{"id":423,"date":"2025-09-20T19:59:58","date_gmt":"2025-09-20T19:59:58","guid":{"rendered":"http:\/\/thomas.goirand.fr\/blog\/?p=423"},"modified":"2025-09-20T19:59:58","modified_gmt":"2025-09-20T19:59:58","slug":"real-time-openstack-packaging-status-with-event-driven-automation","status":"publish","type":"post","link":"http:\/\/thomas.goirand.fr\/blog\/?p=423","title":{"rendered":"Real-Time OpenStack Packaging Status with Event-Driven Automation"},"content":{"rendered":"\n<p>tl;dr: <a href=\"https:\/\/osbpo.debian.net\/deb-status\">https:\/\/osbpo.debian.net\/deb-status<\/a> is now real-time updated and much better than it used to, helping the OpenStack packaging team be a way more efficient.<br><\/p>\n\n\n\n<p><strong>How it used to be<\/strong><\/p>\n\n\n\n<p>For years, the Debian OpenStack team has relied on automated tools to track the status of OpenStack packages across Debian releases. Our goal has always been simple: <strong>transparency, efficiency, accuracy<\/strong>.<\/p>\n\n\n\n<p>We used to use a tool called <code>os-version-checker<\/code>, written by Michal Arbet, which generated a static status page at <a href=\"https:\/\/osbpo.debian.net\/deb-status\">https:\/\/osbpo.debian.net\/deb-status<\/a>. It was functional and served us well \u2014 but it had limitations:<\/p>\n\n\n\n<ul><li>It ran on a cron job, not on demand<\/li><li>It processed all OpenStack releases at once, making it slow<\/li><li>The rsync from Jenkins hosts to osbpo.debian.net was also cron-driven<\/li><li>No immediate feedback after a package build<\/li><\/ul>\n\n\n\n<p>This meant that when a developer pushed a new package to <code>salsa<\/code> (the Debian GitLab instance) in the team&#8217;s repository, the following would happen:<\/p>\n\n\n\n<ul><li>Jenkins would build the backport<\/li><li>Store it in a local repository<\/li><li>Wait up to 30 minutes (or more) for the cron job to run rsync + status update<\/li><li>Only then would the status page reflect the new version<\/li><\/ul>\n\n\n\n<p>For maintainers actively working on a new release, this delay was frustrating. You\u2019d fix a bug, push, build \u2014 and still see your package marked as \u201cmissing\u201d or \u201cout of date\u201d for minutes. You had no real-time feedback. This was also an annoyance for testing, because when fixing a bug, I often had to trigger the rsync manually in order to not wait for it, so I could do my tests. Now, osbpo is always up-to-date a few seconds after the build of the package.<\/p>\n\n\n\n<p><strong>The New Way: Event-Driven, Real-Time Updates<\/strong><\/p>\n\n\n\n<p>We\u2019ve rebuilt the system from the ground up to be fast, responsive, and event-driven. Now, the workflow is:<\/p>\n\n\n\n<ul><li>Developer git push \u2192 triggers Jenkins<\/li><li>Jenkins builds the package \u2192 publishes to local repo<\/li><li>Jenkins immediately triggers a webhook on osbpo.debian.net<\/li><\/ul>\n\n\n\n<p>The webhook on osbpo does:<\/p>\n\n\n\n<ul><li>rsyncs the new package to the central Debian repo<\/li><li>Pulls the latest OpenStack releases from git and use its YAML data (instead of parsing the release HTML pages)<\/li><li>Regenerates the status page, comparing what upstream released and what&#8217;s in Debian<\/li><\/ul>\n\n\n\n<p>No more cron. No more waiting&#8230;<br><br><strong>How it works<\/strong> <\/p>\n\n\n\n<p>The central <code>osbpo.debian.net<\/code> server runs:<\/p>\n\n\n\n<ul><li><code><a href=\"https:\/\/packages.debian.org\/webhook\">webhook<\/a><\/code> \u2014 to receive secure, HMAC-verified triggers that it processes in an async way<\/li><li><code>Apache<\/code> \u2014 to serve the status pages and the Debian OpenStack repositories<\/li><li>Custom scripts \u2014 to <code>rsync<\/code> packages, validate, and generate reports<\/li><\/ul>\n\n\n\n<p>Jenkins instances are configured to <code>curl<\/code> the webhook on successful build. The status page is generated by <code><a href=\"https:\/\/salsa.debian.org\/openstack-team\/debian\/openstack-debian-release-manager\">openstack-debian-release-manager<\/a><\/code>, a new tool I\u2019ve packaged and deployed. The dashboard uses <strong>AJAX<\/strong> to load content dynamically (like when browsing from one release to another), with sorting, metadata, and real-time auto-refresh every 10 seconds.<\/p>\n\n\n\n<p>openstack-debian-release-manager is easy to deploy and configure, and will do most (if not all) of the needed configuration. Uploading it to Debian is probably not needed, and a bit over-kill, so I believe I&#8217;ll just keep it in Salsa for the moment, unless there&#8217;s a way to make it more generic so it can help someone else (another team?) in Debian.<br><\/p>\n\n\n\n<p><strong>Room for improvement<\/strong><\/p>\n\n\n\n<p>There&#8217;s still things I want to add. Namely:<\/p>\n\n\n\n<ul><li>Add status for Debian stable (ie: without the osbpo.debian.net add-on repository), which we used to have with os-version-checker.<\/li><li>Add a per-release config file option to be able to mask not packaged project on a per OpenStack release granularity <\/li><\/ul>\n\n\n\n<p>Special thanks to Michal Arbet for the original os-version-checker that served me for years, helping me to never forget a missing OpenStack package release.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>tl;dr: https:\/\/osbpo.debian.net\/deb-status is now real-time updated and much better than it used to, helping the OpenStack packaging team be a way more efficient. How it used to be For years, the Debian OpenStack team has relied on automated tools to track the status of OpenStack packages across Debian releases. Our goal has always been simple: [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"http:\/\/thomas.goirand.fr\/blog\/index.php?rest_route=\/wp\/v2\/posts\/423"}],"collection":[{"href":"http:\/\/thomas.goirand.fr\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/thomas.goirand.fr\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/thomas.goirand.fr\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/thomas.goirand.fr\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=423"}],"version-history":[{"count":1,"href":"http:\/\/thomas.goirand.fr\/blog\/index.php?rest_route=\/wp\/v2\/posts\/423\/revisions"}],"predecessor-version":[{"id":424,"href":"http:\/\/thomas.goirand.fr\/blog\/index.php?rest_route=\/wp\/v2\/posts\/423\/revisions\/424"}],"wp:attachment":[{"href":"http:\/\/thomas.goirand.fr\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=423"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/thomas.goirand.fr\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=423"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/thomas.goirand.fr\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=423"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}