OpenStack packaging activity, November 2014 to January 2015

November 2014:
Sunday 2nd:
– Travel from Moscow to Paris

Monday 3rd to Sunday 8th:
– Summit in Paris

Monday 10th:
– Uploaded python-rudolf to Sid (needed by Fuel)
– Uploaded python-invoke and python-invocations (needed to run fabric’s unit tests)
– Uploaded python-requests-kerberos/0.5-2 fixing CVE-2014-8650: failure to handle mutual authentication. Asked the release team for unblock.
– Uploaded openstack-pkg-tools version 19 fixing startup with systemd in Jessie (added RuntimeDirectory directive). Asked the release team for unblock.
– Opened ticket to remove TripleO, Tuskar and Ironic packages from Jessie. I don’t consider them ready for a Debian stable release, and there’s no long term support from upstream.
– Fixed Designate Juno dbsync process which prevented it from being installed.
– Fixed Ironic Juno unowned files after purge (policy 6.8, 10.8): /var/lib/ironic/{cache, ironicdb} (eg: purging these folders on purge)

Thuesday 11:
– Fixed nova-api “CVE-2014-3708: Nova network DoS through API filtering” in both the Juno and Icehouse release. Asked the release team to unblock the Icehouse version for Jessie. See: https://bugs.debian.org/769163
– Uploaded Cinder with Duch debconf translation fix and pt.po
– Uploaded python-django-pyscss with upstream patch for Django 1.7 support instead of the Debian one that I wrote 2 months ago. Asked the release team to unblock which they did.

Wednesday 12:
– Uploaded fix for horizon (see #769101) unowned files after purge (policy 6.8, 10.8). Now purging /usr/share/openstack-dashboard/openstack_dashboard on purge.
– Uploaded Ironic with Duch translations of debconf
– Uploaded Designate with Duch translations of Debconf screens
– Uploaded openstack-trove with Duch translations of Debconf screens
– Uploaded Tuskar with Duch translations of Debconf screens
– Updated python-oslotest in Experimental to version 1.2.0

Thursday 13:
– Uploaded new packages: python-oslo.middleware and python-oslo.concurrency.
– Opened a new packaging branch for Nova Kilo, and updated (build-)depends.
– Uploaded fix for Icehouse Cinder: “delete volume failed due to unicode problems”, and asked for unblock.
– Uploaded new package: python-pygit2 and python-xmlbuilder, needed for fuel-agent-ci.
– Uploaded sheepdog with Duch debconf translation.
– Uploaded python-daemonize to Sid (in FTP master NEW queue).
– Re-uploaded python-invoke after FTP master rejection (missing copyright information)

Friday 14:
– Uploaded liberasurecode & python-pyeclib to Sid, now in the FTP masters NEW queue waiting for approval. This will soon be needed by Swift.

Monday 17:
– Worked on the Cobbler packaging (all day long…)

Tuesday 18:
– Worked on backporting all of Fuel packages to Wheezy. Done with fuelclient already.
– Uploaded ruby-cstruct and ruby-rethtool to Sid (needed by nailgun-agent)

Wednesday 19:
– Uploaded pyeclib again, with fixes for the build-depends. Package is still in the NEW queue anyway.
– Built a Debian-based bootstrap hardware discovery image for Fuel, and … it seems that it works already (to be checked…)! \o/
To be added as packages in the ISO:
* nailgun-mcagents
* nailgun-net-check
* fuel-agent
* python-tasklib

Thursday 20:
– Uploaded python-tasklib to Sid (now in NEW queue…)
– Continued working on the discovery bootstrap ISO

Friday 21:
– Documented Sahara procedure in Debian in the official install-guide: https://review.openstack.org/136237
– Fixed oslo.messaging so it doesn’t use PROTOCOL_SSLv3 because its support has been removed from Debian (due to possible protocol downgrade attacks): https://review.openstack.org/136278 and uploaded fixed packages for Sid and Experimental.
– Uploaded fixed Neutron packages for CVE-2014-7821 in both Sid and Experimental (eg: Icehouse and Juno)

Monday 24:
– Uploaded new package: python-os-client-config (in NEW queue)
– Installed new Xen server to be used as my new Jenkins build machine
– Moved the juno-wheezy VM to it
– Finished to package python-pymysql and uploaded to Sid. It’s now running all unit tests successfully! \o/

Tuesday 25:
– Uploaded fix for openstack-debian-images to add the -o compat=1.0 option when building an image with Qemu > 1.0. Opened bug to the release team to have it unblocked.
– Continued working on unit tests for fuel-nailgun.

Wednesday 26:
– Uploaded python-os-net-config to Sid (new package)
– Worked briefly on python-cassandra-driver. It needs cassandra to be in, which is a LOT of work.
– Found a (not useable) hack to run nailgun unit tests. It works, however, it doesn’t seem like fuel-nailgun is designed to be able to use unix socket for the postgres connection in its unit tests.
– Uploaded python-pykmip to Sid (new package)
– Updated the Debian wheezy backport repository for libvirt to version 1.2.9 from official wheezy-backports. Removed policykit-1 and libusb from there too, as it broke stuff to use a backported version (X and usb were not useable on my Wheezy laptop when using it…).

Thursday 27 & Friday 28:
– Uploaded new Javascript packages or dependencies for Fuel: libjs-autonumeric, libjs-backbone-deep-model, libjs-backbone.stickit, libjs-cocktail, libjs-i18next, libjs-require-css, libjs-requirejs, libjs-requirejs-text

Sunday 30:
– Uploaded debian/copyright fixes for libjs-backbone-deep-model, libjs-backbone.stickit and libjs-cocktail after the packages were accepted by the FTP masters and they gave remarks about copyright.

DECEMBER 2014

Monday 01:
– Uploaded new Debian image to MOX, after I unerstood the issue was about the architecture field that I was wrongly filling. I’ll be able to use that for Tempest checking on my dev account.

Tuesday 02:
– Uploaded python-q-text-as-data to Sid (new awesome package!)
– Uploaded Horizon with some triggers mechanisms to start the compress when one of its JS depends is updated. That’s very important for security!
– Uploaded a fixed version of heat-cfntools to Sid (it was missing the /usr/lib/python* folder). Asked the release team for an unblock so it can reach Jessie.
– Fixed unit tests in fuel-nailgun, thanks to a patch from Sebastian Kalinowski. Now all unit tests are passing but one (for which I opened a launchpad bug: tests are trying to write in /var/log/nailgun, which is impossible at package build time).

Wednesday 03:
– Uploaded fixed version of ruby-rethtool after FTP master’s rejection and upstream correction of licensing files.
– Uploaded fixed version of libjs-require-css after FTP master’s rejection
– Fixed (in Git only) python-sysv-ipc missing build-depends on dh-python as per bug opened by James Page (this is not so important, but I did it still).
– Continued working on the tempest-ci scripts.
– Added to the image-guide docs about openstack-debian-images: https://review.openstack.org/#/c/138743/

Thursday 04:
– Uploaded new package: python-proliantutils. Send patch to upstream about an issue in indentation (mix-up with space and tabs) which made the package uninstallable with Python 3.4.

Friday 05:
– Worked on the package CI.

Monday 07:
– Worked on the package CI. All works now, up to all of the Tempest tests for Keystone. Now need to fix the neutron config.

Thuesday 08:
– Continued working on the CI.

Wednesday 09:
– Uploaded fix for FTBFS of python-tasklib (Closes: #772606)
– Uploaded fix for libjerasure-deb missing dependency on libgf-complete-dev, package already unblocked and will migrate to Jessie.
– Uploaded fix for Designate Juno fail to upgrade from Icehouse: this was due to the database_connection directive renamed to connection =.
– Uploaded fix for Designate purge in Sid (Icehouse release of Designate).
– Commited to git updates of the German debconf translation in both Icehouse and Juno.
– Updated nova to use libvirtd as init script dependency instead of libvirt-bin (this was renamed in the libvirt-daemon-system package).
– Do not touch the db connection directive if user didn’t ask for db handling by the package.

Thursday 10 to Saturday 13:
– Finally understood the issues with systemd service files not being activated by default. Fixed openstack-pkg-tools, and uploaded version 20 to Sid, after the release team accepted the changes.

Sunday 14:
– Uploaded Juno 2014.2.1 to Experimental: ceilometer, cinder, glance, python-glance-store, heat, horizon, keystone

Monday 15:
– Finished uploading Juno 2014.2.1 to Experimental: Nova, Neutron, Sahara

Tuesday 16:
– Added crontab to flush tokens in Icehouse Keystone
– Some more CI work

Wednesday 17:
– Uploaded keystone with systemd fix and crontab to flush the token table in Sid (eg: Icehouse).
– Uploaded nova Icehouse with a bunch of fixes in Sid.

Thursday 18:
– Updated some issues in Nova Icehouse (Sid/Jessie)

Friday 19:
– Started building a new Jenkins instance for building Kilo packages

Monday 22:
– Finished building the new Jenkins instance for building Kilo packages, and rebuilt every packages there, using Jessie as a base.

Tuesday 23:
– Updated version for the following packages: oslo.utils, oslo.middleware, stevedore, oslo.concurency, pecan, oslo.concurrency, python-oslo.vmware, python-glance-store
– Built so far: Ceilometer, Keystone, python-glanceclient, cinder, glance

Wednesday 24:
– Continued packaging Kilo beta 1. Updated: nova, designate, neutron
– Uploaded python-tempest-lib to Debian Unstable (new package)

Wednesday 31:
– Continued packaging Kilo beta 1. Updated: heat

JANUARY 2015

Thursday 01:
– Continued packaging Kilo beta 1. Updated: ironic, openstack-trove, openstack-doc-tools, ceilometer

Friday 02:
– Finished packaging Kilo beta 1. Updated: Sahara, Murano, Murano-dashboard, Murano-agent

Sunday 04:
– Started testing Kilo beta 1. Fixed a few issues on default configuration for Ceilometer and Glance.

Monday 05:
– Fixed openstack-pkg-tools which failed to create PID files at boot time, Uploaded to Sid, asked the release team for unblock.
– Uploaded ceilometer & cinder to Sid, rebuilt against openstack-pkg-tools 21.
– Did more testing of Kilo beta 1, fixed a few more minor issues.

Tuesday 06:
– Uploaded glance, neutron, nova, designate, keystone, heat, trove to Sid, so that all sysv-rc init scripts are fixed with the new openstack-pkg-tools 21. Designate, heat, keystone and trove contains other minor fixes reported to the Debian BTS.

Wednesday 07:
– Asked the Debian release team (open bugs with debdiff as attachment) for unblocks of glance, neutron, nova, designate, keystone, heat, trove so they migrate to Jessie.
– Fixed a few minor issues tracked in the Debian BTS on various packages.

Thesday 08:
– James Page from Canonical informed me that they are now using openstack-pkg-tools for maintaining their daemons in Nova, Cinder and Keystone in Ubuntu. That’s an awesome news : more QA for both platforms.
– James Page found out that dh_installinit *must* be called *after* the call of dh_systemd_enable, otherwise, daemons aren’t started automatically at the first install of packages, as the unmask of systemd happens after the invoke-rc.d.

Friday 09:
– Did some QA checks on the latest upload. Fixed Heat which broke because using the wrong template name (glance instead of heat).

Monday 12:
– Started re-running the automated openstack-deploy scrip in Icehouse, Juno and Kilo. Found out the issue in Keystone wasn’t fixed in Juno (but was fixed in other releases), and fixed it.
– Removed the use of ssl.PROTOCOL_SSLv3 from heat (removed form Debian). Uploaded the fixed package to Sid.
– All of openstack-deploy (debian/kilo branch) now works and succesfully installs OpenStack again.

If dh_installinit is called before, we have:

# Automatically added by dh_installinit
if [ -x "/etc/init.d/keystone" ]; then
update-rc.d keystone defaults >/dev/null
fi
if [ -x "/etc/init.d/keystone" ] || [ -e "/etc/init/keystone.conf" ]; then
invoke-rc.d keystone start || true
fi
# End automatically added section
# Automatically added by dh_systemd_enable
# This will only remove masks created by d-s-h on package removal.
deb-systemd-helper unmask keystone.service >/dev/null || true

# was-enabled defaults to true, so new installations run enable.
if deb-systemd-helper --quiet was-enabled keystone.service; then
# Enables the unit on first installation, creates new
# symlinks on upgrades if the unit file has changed.
deb-systemd-helper enable keystone.service >/dev/null || true
else
# Update the statefile to add new symlinks (if any), which need to be
# cleaned up on purge. Also remove old symlinks.
deb-systemd-helper update-state keystone.service >/dev/null || true
fi
# End automatically added section

If it’s called after dh_systemd_enable, we have:

# Automatically added by dh_systemd_enable
# This will only remove masks created by d-s-h on package removal.
deb-systemd-helper unmask keystone.service >/dev/null || true

# was-enabled defaults to true, so new installations run enable.
if deb-systemd-helper --quiet was-enabled keystone.service; then
# Enables the unit on first installation, creates new
# symlinks on upgrades if the unit file has changed.
deb-systemd-helper enable keystone.service >/dev/null || true
else
# Update the statefile to add new symlinks (if any), which need to be
# cleaned up on purge. Also remove old symlinks.
deb-systemd-helper update-state keystone.service >/dev/null || true
fi
# End automatically added section
# Automatically added by dh_installinit
if [ -x "/etc/init.d/keystone" ]; then
update-rc.d keystone defaults >/dev/null
fi
if [ -x "/etc/init.d/keystone" ] || [ -e "/etc/init/keystone.conf" ]; then
invoke-rc.d keystone start || true
fi
# End automatically added section

As a consequence, I have to re-upload version 22 of openstack-pkg-tools and also re-upload all OpenStack core packages to Debian Sid.

– Fixed a number of issues like:
* dbc_upgrade = true check which shouldn’t have been there in postinst.
* <project>/configure_db default value is now always false
* db_sync and pkgos_dbc_postinst are now only done if <project>/configure_db is set to true.
– Rebuilt all packages in Juno and Kilo with the above changes.

Tuesday 13:
– Opened unblock bugs for the release team to unblock all fixed packages.
– Made more tests in Juno and Kilo to make sure the fixed bugs in Icehouse are fixed there too.
– Fixed numerous issues in Trove (missing trove-conductor.conf, wrong trove-api init file, etc.). More work will be needed for it for both Icehouse and newer releases.

Wednesday 14:
– Did a doc meeting about debconf. Some doc contributors still want to kill the debconf / debian manual, and I have to not agree.
– Made a new patch to better document the keystone install procedure:
– Did some bug triaging in the doc about Debian.
– Uploaded new versions of core packages to Experimental (eg: Juno) built against openstack-pkg-tools >= 22~, and some fixes forward ported from Icehouse: Keystone, Ceilometer, Cinder, Glance, Heat, Ironic, Murano, Neutron, Nova, Saraha and Murano-agent. All where rebuilt in Juno (Wheezy + Trusty) and Kilo (Jessie only) on my Jenkins.

Thuesday 15:
– Succesfully booted a live-build Debian live image containing mcollective and nailgun-agent as a Debian replacement for the hardware discovery / boostrap image of Fuel. Now, I need to find a way to use just a kernel + initramfs

Friday 16 to Tuesday 20:
– Worked on the packaging CI.

Wednesday 21:
– Fixed https://bugs.debian.org/775636 (Horizon failed to build due to a Moscow timezone change and wrong test). Uploaded to Sid, asked for unblock.
– Fixed https://bugs.debian.org/775926: CVE-2015-1195: Glance still allows users to download and delete any file in glance-api server (applied upstream patch). Uploaded to Sid, asked for unblock. Uploaded Juno version to Experimental.
– Uploaded openstack-trove with the remaining fixes, asked release team for unblock.
– Uploaded python-glanceclient 0.15.0 (Juno) to Experimental because it fixes an issue with HTTPS. Added to it a patch from James Page not yet merged, which fixes unit test with Python 2.7.9 (7 failures otherwise).
– Uploaded python-xstatic-d3 as it can’t be installed anymore in Sid after a new version of d3 was uploaded.

Thursday 22:
– Uploaded python-xstatic-smart-table and libjs-angularjs-smart-table to Sid (new packages, now in NEW queue).

Friday 23:
– Ask for the removal of the below list of packages from Jessie:
python-xstatic
python-xstatic-angular
python-xstatic-angular-cookies
python-xstatic-angular-mock
python-xstatic-bootstrap-datepicker
python-xstatic-bootstrap-scss
python-xstatic-d3
python-xstatic-font-awesome
python-xstatic-hogan
python-xstatic-jasmine
python-xstatic-jquery
python-xstatic-jquery-migrate
python-xstatic-jquery-ui
python-xstatic-jquery.bootstrap.wizard
python-xstatic-jquery.quicksearch
python-xstatic-jquery.tablesorter
python-xstatic-jsencrypt
python-xstatic-qunit
python-xstatic-rickshaw
python-xstatic-spin
libjs-jsencrypt
libjs-spin.js
libjs-twitter-bootstrap-datepicker
libjs-twitter-bootstrap-wizard

They are used only in OpenStack Horizon starting on 2014.2 (aka Juno), and Jessie is shipped with Icehouse, so it’s IMO best to not carry the burden of maintaining these packages for the life of Jessie.

Monday 26:
– Enhanced and review requested changes for https://review.openstack.org/147296 (ie: Keystone install with more details about what the package does).
– Finished testing network on the CI install. Now need to automate all.

Tuesday 27:
– Closed all bugs on the rabbitmq-server package (2 correction, one bug triage).
– Uploaded a fix for the missing conntrack dependency in neutron-l3-agent.
– Restarted working on CI setup of Juno after success with manual install in a Xen domU.
– Uploaded fix to make sheepdog build reproducible (patch from the Debian BTS).

Thursday 28:
– Fixed and uploaded to Sid openstack-debian-images 2 bugs reported by Steve McIntire. Official Debian images for OpenStack are now available at:
http://cdimage.debian.org/cdimage/openstack/ \o/
Note that this is the weekly build of testing. We wont get Debian Stable images before Jessie is out.
– Documented the new image thing here: http://docs.openstack.org/image-guide/content/ch_obtaining_images.html#debian-images as a new patch: https://review.openstack.org/#/c/151015/
– Fixed my patch for keystone debconf doc at: https://review.openstack.org/#/c/147296/

Wednesday 29:
– Continued working on packaging CI

Thursday 30:
– Fixed CVE on Neutron (Juno): L3 agent denial of service with radvd 2.0+
– Fixed CVE on Glance (Icehouse + Juno): Glance user storage quota bypass. Asked release team for unblock.
– Fixed the image-guide patch after review (ie: https://review.openstack.org/151015)