{"id":61458,"date":"2021-05-18T19:03:32","date_gmt":"2021-05-18T16:03:32","guid":{"rendered":"https:\/\/www.altoros.com\/blog\/?p=61458"},"modified":"2021-11-10T01:56:46","modified_gmt":"2021-11-09T22:56:46","slug":"adobe-migrates-5500-services-to-kubernetes","status":"publish","type":"post","link":"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/","title":{"rendered":"Adobe Migrates 5,500+ Services to Kubernetes"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_79_2 counter-hierarchy ez-toc-counter ez-toc-transparent ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#Reaching_feature_parity_on_Kubernetes\" >Reaching feature parity on Kubernetes<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#Conducting_a_live_migration\" >Conducting a live migration<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#Creating_dedicated_infrastructure\" >Creating dedicated infrastructure<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#Reducing_the_impact_of_failure\" >Reducing the impact of failure<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#Want_details_Watch_the_videos\" >Want details? Watch the videos!<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#Further_reading\" >Further reading<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#About_the_experts\" >About the experts<\/a><\/li><\/ul><\/nav><\/div>\n<h3><span class=\"ez-toc-section\" id=\"Reaching_feature_parity_on_Kubernetes\"><\/span>Reaching feature parity on Kubernetes<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>In 2015, Adobe began <a href=\"https:\/\/medium.com\/adobetech\/green-computing-scaling-adobe-experience-platform-resources-for-sustainable-business-50f00d022f12\" rel=\"noopener noreferrer\" target=\"_blank\">Ethos<\/a>, a project to consolidate infrastructure and software development best practices in the company. The project started with the adoption of Docker containers built on top of Apache Mesos. This led to the first containerized production services of Adobe.<\/p>\n<p>Needing to scale up, Adobe switched from Apache Mesos to the Distributed Cloud Operating System (DC\/OS) in late 2016 to early 2017. With DC\/OS, Adobe was able to scale up with a multitenant platform. However, to meet with the compliance, scaling, and security requirements of a multitenant platform, Adobe&#8217;s engineering team needed to build their own components for crucial features, such as log forwarding, monitoring, autoscaling, container security frameworks, etc.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Live-Kubernetes-Ethos-History.png\"><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Live-Kubernetes-Ethos-History-1024x576.png\" alt=\"\" width=\"640\" class=\"aligncenter size-large wp-image-61462\" \/><\/a><small>Progression of the Ethos project (<a href=\"https:\/\/www.youtube.com\/watch?v=HVmE-GD1rTs\" rel=\"noopener noreferrer\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<p>In 2018, Adobe started a <a href=\"https:\/\/en.wikipedia.org\/wiki\/Skunkworks_project\" rel=\"noopener noreferrer\" target=\"_blank\">skunkworks project<\/a>, which identified Kubernetes reached feature parity with the custom components created by the in-house team. This greenlit a complete migration to Kubernetes. &#8220;We reached a point where it wasn&#8217;t a question of <i>if<\/i>, but <i>when<\/i> are we going to make the jump to Kubernetes?&#8221; explained <a href=\"https:\/\/www.linkedin.com\/in\/isom\/\" rel=\"noopener noreferrer\" target=\"_blank\">Sean Isom<\/a>, Engineering Manager at Adobe.<\/p>\n<div id=\"attachment_61472\" style=\"width: 130px\" class=\"wp-caption alignright\"><img decoding=\"async\" aria-describedby=\"caption-attachment-61472\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Sean-Isom-150x150.jpg\" alt=\"\" width=\"120\" class=\"size-thumbnail wp-image-61472\" \/><p id=\"caption-attachment-61472\" class=\"wp-caption-text\"><small>Sean Isom<\/small><\/p><\/div>\n<blockquote><p>&#8220;We needed to quickly execute on being able to move all of our existing DC\/OS services to our new Kubernetes clusters and then end-of-life our DC\/OS stack. These are all production services, and we had to execute this move across clusters seamlessly without downtime, so users would not notice an impact to their service.&#8221; <\/p>\n<p>\u2014Sean Isom, Adobe<\/p><\/blockquote>\n<p>By 2019, Adobe had successfully onboarded new services to Kubernetes. The company then started the process of migrating the rest of its existing services.<\/p>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Conducting_a_live_migration\"><\/span>Conducting a live migration<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>To help streamline the migration process, Adobe relied on the Ethos service spec, a YAML file that represents a service&#8217;s requirements. The Ethos service spec is not limited to either DC\/OS or Kubernetes. It specifies what a service&#8217;s environments are, as well as what resources it needs.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Live-Kubernetes-Ethos-Service-spec.png\"><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Live-Kubernetes-Ethos-Service-spec-1024x576.png\" alt=\"\" width=\"640\" class=\"aligncenter size-large wp-image-61476\" \/><\/a><small>An example of an Ethos service spec (<a href=\"https:\/\/www.youtube.com\/watch?v=HVmE-GD1rTs\" rel=\"noopener noreferrer\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<blockquote><p>&#8220;The service spec has become our single source of truth. It informs all the infrastructure we provision. From our perspective, when we migrate or add a new cluster orchestrator, it&#8217;s a matter of building a functionally equivalent transformation of the service spec into the new architecture. This level of abstraction allows us to do that.&#8221; \u2014Sean Isom, Adobe<\/p><\/blockquote>\n<p>The Adobe engineering team prepared a migration tool comprising three steps:<\/p>\n<ol>\n<li style=\"margin-bottom: 6px;\"><b>Configure and deploy.<\/b> Services are configured to seem like they were initially onboarded to Kubernetes. Services are then deployed to a temporary DNS endpoint to allow health checks to run automatically.<\/li>\n<li style=\"margin-bottom: 6px;\"><b>DNS cutover.<\/b> Once all health checks are passed, a user can simply flip the service from DC\/OS to Kubernetes.<\/li>\n<li><b>Mark as good.<\/b> After the service has been fully tested on Kubernetes, they can be marked as good. This results in the removal of infrastructure used by the service on DC\/OS.<\/li>\n<\/ol>\n<p>The tool was exposed to users, which are developers of Adobe services, through a self-service portal, enabling them to migrate their own services. At any point during migration, a service can rollback in case an error occurs.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Live-Kubernetes-Migration-Tooling-self-service-bulk.png\"><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Live-Kubernetes-Migration-Tooling-self-service-bulk-1024x576.png\" alt=\"\" width=\"640\" class=\"aligncenter size-large wp-image-61484\" \/><\/a><small>Service migration in three stages (<a href=\"https:\/\/www.youtube.com\/watch?v=HVmE-GD1rTs\" rel=\"noopener noreferrer\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<p>With thousands of services and users still on DC\/OS, the Adobe engineering team built a bulk migration tool, which automated the entire process. For example, bulk migration can be used to configure and deploy all the services in a cluster. All services that fail are rolled back, while those that succeed proceed to the next step.<\/p>\n<p>During migration, users are notified of the status of their services.<\/p>\n<blockquote><p>&#8220;It&#8217;s an important dichotomy to give users the flexibility to run the migration themselves and to roll things back, but also to keep control on our end to be able to perform mass migration on clusters, so we&#8217;re not constantly waiting on users to take action.&#8221; \u2014Sean Isom, Adobe<\/p><\/blockquote>\n<p>In the early days of the Ethos project, the CI\/CD system was built separately from the cluster infrastructure. This removed the need to make structural changes to the provisioning system or to the CI\/CD system during the migration.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Live-Kubernetes-Migration-Deployment-Architecture.png\"><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Live-Kubernetes-Migration-Deployment-Architecture-1024x576.png\" width=\"640\" class=\"aligncenter size-large wp-image-61464\" \/><\/a><small>The Ethos deployment pipeline (<a href=\"https:\/\/www.youtube.com\/watch?v=HVmE-GD1rTs\" rel=\"noopener noreferrer\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<p>The Adobe engineering team could simply be able to replace the components that interacted with the cluster and did the deployment. For example, in DC\/OS, the Ethos service spec would be translated to a Marathon JSON file or as a YAML file in Kubernetes.<\/p>\n<p>Next, the team created a migration schedule that was flexible, but also realistic. Throughout the migration period, users were given the ability to use the self-service tools. In addition, managed bulk migrations were used on clusters on a per-wave basis going from environment to environment.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/adobe-performing-migration-to-kubernetes-in-waves.png\"><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/adobe-performing-migration-to-kubernetes-in-waves-1024x576.png\" alt=\"\" width=\"640\" class=\"aligncenter size-large wp-image-61535\" \/><\/a><small>Performing migration in waves (<a href=\"https:\/\/www.youtube.com\/watch?v=HVmE-GD1rTs\" rel=\"noopener noreferrer\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<blockquote><p>&#8220;We would bulk migrate a stage cluster once, wait a few weeks, and then retry what did not migrate successfully the first time. This is a pattern that we built over time, where we migrate what&#8217;s there. Anything that succeeded automatically moved on. Anything that failed, or a user rolled back, would get left behind. We&#8217;d work with the users to iterate on those services, fix what we could, and try again. This was the key strategy that allowed us to get the migration done in less than a year.&#8221;<br \/>\n\u2014Sean Isom, Adobe<\/p><\/blockquote>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Creating_dedicated_infrastructure\"><\/span>Creating dedicated infrastructure<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>One of the products running on Kubernetes is Adobe Experience Manager (AEM), a content management system used by multiple Fortune 100 organizations. &#8220;Adobe Experience Manager runs on Microsoft Azure,&#8221; noted <a href=\"https:\/\/www.linkedin.com\/in\/carlossg\/\" rel=\"noopener noreferrer\" target=\"_blank\">Carlos Sanchez<\/a>, Senior Cloud Software Engineer at Adobe. &#8220;We have more than 14 clusters right now and are growing,&#8221; he said in May 2021 at KubeCon Europe.<\/p>\n<div id=\"attachment_61551\" style=\"width: 130px\" class=\"wp-caption alignright\"><img decoding=\"async\" aria-describedby=\"caption-attachment-61551\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Carlos-Sanchez-150x150.jpg\" alt=\"\" width=\"120\" class=\"size-thumbnail wp-image-61551\" \/><p id=\"caption-attachment-61551\" class=\"wp-caption-text\"><small>Carlos Sanchez<\/small><\/p><\/div>\n<p>While the move to Kubernetes added multitenancy capabilities for the clusters, it also brought a new challenge.<\/p>\n<blockquote><p>&#8220;The requirement we got from customers is that they wanted to have dedicated infrastructure. This boiled down to dedicated egress IP addresses, private connections, and virtual private networks.&#8221; \u2014Carlos Sanchez, Adobe<\/p><\/blockquote>\n<p>In order to provide dedicated infrastructure for each customer, the AEM team built a solution using <a href=\"http:\/\/www.squid-cache.org\/\" rel=\"noopener noreferrer\" target=\"_blank\">Squid<\/a>, a caching and forwarding HTTP web proxy.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Experience-Manager-Kubernetes-dedicated-infrastructure-squid.png\"><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Experience-Manager-Kubernetes-dedicated-infrastructure-squid-1024x576.png\" alt=\"\" width=\"640\" class=\"aligncenter size-large wp-image-61553\" \/><\/a><small>Dedicated infrastructure using Squid (<a href=\"https:\/\/static.sched.com\/hosted_files\/kccnceu2021\/66\/Dedicated%20Infrastructure%20in%20a%20Multitenant%20World.pdf\" rel=\"noopener noreferrer\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<p>Squid provided a couple of upsides in that:<\/p>\n<ul>\n<li style=\"margin-bottom: 6px;\">Java virtual machine configuration is simple and transparent.<\/li>\n<li>Virtual network peering makes all traffic private.<\/li>\n<\/ul>\n<p>On the other hand, there were also some downsides:<\/p>\n<ul>\n<li style=\"margin-bottom: 6px;\">Proxy authentication and authorization is not well supported.<\/li>\n<li style=\"margin-bottom: 6px;\">Squid only works on the HTTP and HTTPS protocols.<\/li>\n<li style=\"margin-bottom: 6px;\">HTTP traffic is not encrypted.<\/li>\n<li>Squid does not support virtual private networks or other private connections.<\/li>\n<\/ul>\n<p>Due to the limitations with Squid, the AEM team looked for another solution. In the second iteration, the team opted to use <a href=\"https:\/\/www.envoyproxy.io\/\" rel=\"noopener noreferrer\" target=\"_blank\">Envoy<\/a>.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Experience-Manager-Kubernetes-dedicated-infrastructure-envoy.png\"><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Experience-Manager-Kubernetes-dedicated-infrastructure-envoy-1024x576.png\" alt=\"\" width=\"640\" class=\"aligncenter size-large wp-image-61559\" \/><\/a><small>Dedicated infrastructure using Envoy (<a href=\"https:\/\/static.sched.com\/hosted_files\/kccnceu2021\/66\/Dedicated%20Infrastructure%20in%20a%20Multitenant%20World.pdf\" rel=\"noopener noreferrer\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<p>The iteration with Envoy provided the following advantages:<\/p>\n<ul>\n<li style=\"margin-bottom: 6px;\">Java virtual machine configuration is simple and transparent.<\/li>\n<li style=\"margin-bottom: 6px;\">Any protocol can be supported using different listeners in Envoy sidecars.<\/li>\n<li style=\"margin-bottom: 6px;\">All traffic is encrypted.<\/li>\n<li style=\"margin-bottom: 6px;\">The virtual network provides multiple configuration options on the service level of the cloud provider.<\/li>\n<li>mTLS prevents unauthorized connections.<\/li>\n<\/ul>\n<p>Meanwhile, it also had certain disadvantages:<\/p>\n<ul>\n<li style=\"margin-bottom: 6px;\">Virtual private networks and other private connections require a non-overlapping IP range with a customer&#8217;s private network.<\/li>\n<li>Certificate management can be complex.<\/li>\n<\/ul>\n<blockquote><p>&#8220;We run Envoy on the virtual machines, as well as on pod sidecars in Kubernetes.&#8221; \u2014Carlos Sanchez, Adobe<\/p><\/blockquote>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Reducing_the_impact_of_failure\"><\/span>Reducing the impact of failure<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>To minimize the number of errors during migration, Adobe&#8217;s engineering team prepared several <b>precautions<\/b>. First, the migration tool was feature-flagged. This enabled the team to pause the progress of migration if a problem is detected, providing the opportunity to troubleshoot.<\/p>\n<p>Second, the waves in managed bulk migrations are designed for active pauses. This means that bulk migration can be stopped, while allowing the self-service tool to still function. This way, users who have no issues can complete migration, while the Adobe team can focus on issues such as cluster stability.<\/p>\n<p>Third, rollbacks are expected and even automated. As previously mentioned, during the configure and deploy step of the migration, health checks are performed on the service and, if any of them fail, the service is automatically rolled back.<\/p>\n<p><center><a href=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Live-Kubernetes-migration-guardrails.png\"><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Adobe-Live-Kubernetes-migration-guardrails-1024x576.png\" width=\"640\" class=\"aligncenter size-large wp-image-61504\" \/><\/a><small>Errors should be expected and prepared for (<a href=\"https:\/\/www.youtube.com\/watch?v=HVmE-GD1rTs\" rel=\"noopener noreferrer\" target=\"_blank\">Image credit<\/a>)<\/small><\/center><\/p>\n<blockquote><p>&#8220;We needed to make sure that we were prepared to fail gracefully during migration. Not everything is going to come up successfully on Kubernetes the first time. Sometimes, services will exhibit interesting artifacts that only become known after they&#8217;ve been running and serving production traffic on Kubernetes. We had to make sure that every level of our design had flexibility built into it.&#8221;<br \/>\n\u2014Sean Isom, Adobe<\/p><\/blockquote>\n<p>To conclude, Sean shared some of the <b>lessons learned<\/b> during migration.<\/p>\n<ul>\n<li style=\"margin-bottom: 6px;\">Abstractions, like the Ethos service spec, were critical in during the process of migration, enabling the company to transition from DC\/OS to Kubernetes fairly quickly.<\/li>\n<li style=\"margin-bottom: 6px;\">Always have your features in a consistent state regardless of the platform.<\/li>\n<li style=\"margin-bottom: 6px;\">Empathize with users and keep them engaged during the migration process.<\/li>\n<li>Be firm on the migration goals, but be flexible on the execution.<\/li>\n<\/ul>\n<blockquote><p>&#8220;We made sure that users understood why we had to migrate, but also we understood what their requirements were for the platform. It&#8217;s important to have that level of mutual empathy for the process.&#8221; \u2014Sean Isom, Adobe<\/p><\/blockquote>\n<p>With proper tooling, guardrails, as well as a schedule in place, the Adobe engineering team managed to completely migrate all of the existing DC\/OS services to Kubernetes in 2020. The entire process lasted nine months, and Adobe was able to migrate over 5,500 services, more than 1,200 of which serve in production. All DC\/OS clusters have also been decommissioned.<\/p>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Want_details_Watch_the_videos\"><\/span>Want details? Watch the videos!<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><small>In the video below, <a href=\"https:\/\/www.linkedin.com\/in\/isom\/\" rel=\"noopener noreferrer\" target=\"_blank\">Sean Isom<\/a> details how Adobe managed to perform a live migration of its services to Kubernetes.<\/small><\/p>\n<p><center><iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/HVmE-GD1rTs\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/center><\/p>\n<p>&nbsp;<br \/>\n<small>In this next video, Carlos Sanchez discusses how the Adobe Experience Manager team built solutions to provide customers with dedicated infrastructure.<\/small><\/p>\n<p><center><iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/Kvog7aZ3esY\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/center><\/p>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Further_reading\"><\/span>Further reading<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul>\n<li><a href=\"https:\/\/www.altoros.com\/blog\/github-crafts-10-custom-kubernetes-controllers-to-refine-provisioning\/\">GitHub Crafts 10+ Custom Kubernetes Controllers to Refine Provisioning<\/a><\/li>\n<li><a href=\"https:\/\/www.altoros.com\/blog\/linkedin-aims-to-deploy-thousands-of-hadoop-servers-on-kubernetes\/\">LinkedIn Aims to Deploy Thousands of Hadoop Servers on Kubernetes<\/a><\/li>\n<li><a href=\"https:\/\/www.altoros.com\/blog\/maturity-of-app-deployments-on-kubernetes-from-manual-to-automated-ops\/\">Maturity of App Deployments on Kubernetes: From Manual to Automated Ops<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h3><span class=\"ez-toc-section\" id=\"About_the_experts\"><\/span>About the experts<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Sean-Isom-bio.png\" alt=\"\" width=\"120\" class=\"alignright size-full wp-image-61508\" \/><\/p>\n<p><small><a href=\"https:\/\/www.linkedin.com\/in\/isom\/\" rel=\"noopener noreferrer\" target=\"_blank\">Sean Isom<\/a> is Engineering Manager at Adobe, where he works on the Ethos project, the company&#8217;s container compute, and the CI\/CD framework. He is responsible for the Kubernetes migration project, live migrating over 5,000 service environments from DC\/OS to Adobe&#8217;s own Kubernetes implementation. Previously, Sean worked as a senior engineer and manager at several startups and medium-sized businesses, transitioning into cloud engineering after working in computer graphics and with C++.<\/small><\/p>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/Carlos-Sanchez-bio.png\" width=\"120\" class=\"alignright size-full wp-image-61566\" \/><\/p>\n<p><small><a href=\"https:\/\/www.linkedin.com\/in\/carlossg\/\" rel=\"noopener noreferrer\" target=\"_blank\">Carlos Sanchez<\/a> is Senior Cloud Software Engineer at Adobe, where he specializes in software automation\u2014from build tools to continuous delivery and progress delivery. For more than a decade, Carlos has been a member of the Apache Software Foundation, contributing to numerous open-source projects, such as Maven, ActiveMQ, Jenkins, Puppet, Fog, Commons, etc. During his career, he was also an engineer and a consultant in various technology companies, as well as a project lead at the Eclipse Foundation.<\/small><\/p>\n<hr\/>\n<p><center><small>This blog post was written by <a href=\"https:\/\/www.altoros.com\/blog\/author\/carlo\/\">Carlo Gutierrez<\/a>, edited by <a href=\"https:\/\/www.altoros.com\/blog\/author\/sophie.turol\/\">Sophia Turol<\/a> and <a href=\"https:\/\/www.altoros.com\/blog\/author\/alex\/\">Alex Khizhniak<\/a>.<\/small><\/center><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Reaching feature parity on Kubernetes<\/p>\n<p>In 2015, Adobe began Ethos, a project to consolidate infrastructure and software development best practices in the company. The project started with the adoption of Docker containers built on top of Apache Mesos. This led to the first containerized production services of Adobe.<\/p>\n<p>Needing to scale up, [&#8230;]<\/p>\n","protected":false},"author":32,"featured_media":61577,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":"","_links_to":"","_links_to_target":""},"categories":[26],"tags":[873,815,912],"class_list":["post-61458","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-use-cases","tag-cloud-native","tag-digital-transformation","tag-kubernetes"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.6 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Adobe Migrates 5,500+ Services to Kubernetes | Altoros<\/title>\n<meta name=\"description\" content=\"With a detailed plan, including tooling, guardrails, and a roadmap, the company successfully migrated thousands of its services to Kubernetes in 9 months.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Adobe Migrates 5,500+ Services to Kubernetes | Altoros\" \/>\n<meta property=\"og:description\" content=\"Reaching feature parity on Kubernetes In 2015, Adobe began Ethos, a project to consolidate infrastructure and software development best practices in the company. The project started with the adoption of Docker containers built on top of Apache Mesos. This led to the first containerized production services of Adobe. Needing to scale up, [...]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/\" \/>\n<meta property=\"og:site_name\" content=\"Altoros\" \/>\n<meta property=\"article:published_time\" content=\"2021-05-18T16:03:32+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-11-09T22:56:46+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/adobe-migrates-thousands-of-services-to-kubernetes.gif\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"576\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/gif\" \/>\n<meta name=\"author\" content=\"Carlo Gutierrez\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Carlo Gutierrez\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/\",\"url\":\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/\",\"name\":\"Adobe Migrates 5,500+ Services to Kubernetes | Altoros\",\"isPartOf\":{\"@id\":\"https:\/\/www.altoros.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/adobe-migrates-thousands-of-services-to-kubernetes.gif\",\"datePublished\":\"2021-05-18T16:03:32+00:00\",\"dateModified\":\"2021-11-09T22:56:46+00:00\",\"author\":{\"@id\":\"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/833e109f77de753b2b472dca0236b442\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#primaryimage\",\"url\":\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/adobe-migrates-thousands-of-services-to-kubernetes.gif\",\"contentUrl\":\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/adobe-migrates-thousands-of-services-to-kubernetes.gif\",\"width\":1024,\"height\":576},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.altoros.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Adobe Migrates 5,500+ Services to Kubernetes\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.altoros.com\/blog\/#website\",\"url\":\"https:\/\/www.altoros.com\/blog\/\",\"name\":\"Altoros\",\"description\":\"Insight\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.altoros.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/833e109f77de753b2b472dca0236b442\",\"name\":\"Carlo Gutierrez\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/02\/CG_portrait-2-96x96.jpg\",\"contentUrl\":\"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/02\/CG_portrait-2-96x96.jpg\",\"caption\":\"Carlo Gutierrez\"},\"description\":\"Carlo Gutierrez is a Technical Writer at Altoros. As part of the editorial team, his focus has been on emerging technologies such as Cloud Foundry, Kubernetes, blockchain, and the Internet of Things. Prior to Altoros, he primarily wrote about enterprise and consumer technology. Carlo has over 12 years of experience in the publishing industry. Previously, he served as an Editor for PC World Philippines and Questex Asia, as well as a Designer for Tropa Entertainment.\",\"url\":\"https:\/\/www.altoros.com\/blog\/author\/carlo\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Adobe Migrates 5,500+ Services to Kubernetes | Altoros","description":"With a detailed plan, including tooling, guardrails, and a roadmap, the company successfully migrated thousands of its services to Kubernetes in 9 months.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/","og_locale":"en_US","og_type":"article","og_title":"Adobe Migrates 5,500+ Services to Kubernetes | Altoros","og_description":"Reaching feature parity on Kubernetes In 2015, Adobe began Ethos, a project to consolidate infrastructure and software development best practices in the company. The project started with the adoption of Docker containers built on top of Apache Mesos. This led to the first containerized production services of Adobe. Needing to scale up, [...]","og_url":"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/","og_site_name":"Altoros","article_published_time":"2021-05-18T16:03:32+00:00","article_modified_time":"2021-11-09T22:56:46+00:00","og_image":[{"width":1024,"height":576,"url":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/adobe-migrates-thousands-of-services-to-kubernetes.gif","type":"image\/gif"}],"author":"Carlo Gutierrez","twitter_misc":{"Written by":"Carlo Gutierrez","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/","url":"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/","name":"Adobe Migrates 5,500+ Services to Kubernetes | Altoros","isPartOf":{"@id":"https:\/\/www.altoros.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#primaryimage"},"image":{"@id":"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#primaryimage"},"thumbnailUrl":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/adobe-migrates-thousands-of-services-to-kubernetes.gif","datePublished":"2021-05-18T16:03:32+00:00","dateModified":"2021-11-09T22:56:46+00:00","author":{"@id":"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/833e109f77de753b2b472dca0236b442"},"breadcrumb":{"@id":"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#primaryimage","url":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/adobe-migrates-thousands-of-services-to-kubernetes.gif","contentUrl":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/05\/adobe-migrates-thousands-of-services-to-kubernetes.gif","width":1024,"height":576},{"@type":"BreadcrumbList","@id":"https:\/\/www.altoros.com\/blog\/adobe-migrates-5500-services-to-kubernetes\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.altoros.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Adobe Migrates 5,500+ Services to Kubernetes"}]},{"@type":"WebSite","@id":"https:\/\/www.altoros.com\/blog\/#website","url":"https:\/\/www.altoros.com\/blog\/","name":"Altoros","description":"Insight","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.altoros.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/833e109f77de753b2b472dca0236b442","name":"Carlo Gutierrez","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.altoros.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/02\/CG_portrait-2-96x96.jpg","contentUrl":"https:\/\/www.altoros.com\/blog\/wp-content\/uploads\/2021\/02\/CG_portrait-2-96x96.jpg","caption":"Carlo Gutierrez"},"description":"Carlo Gutierrez is a Technical Writer at Altoros. As part of the editorial team, his focus has been on emerging technologies such as Cloud Foundry, Kubernetes, blockchain, and the Internet of Things. Prior to Altoros, he primarily wrote about enterprise and consumer technology. Carlo has over 12 years of experience in the publishing industry. Previously, he served as an Editor for PC World Philippines and Questex Asia, as well as a Designer for Tropa Entertainment.","url":"https:\/\/www.altoros.com\/blog\/author\/carlo\/"}]}},"_links":{"self":[{"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/posts\/61458","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/users\/32"}],"replies":[{"embeddable":true,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/comments?post=61458"}],"version-history":[{"count":106,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/posts\/61458\/revisions"}],"predecessor-version":[{"id":64805,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/posts\/61458\/revisions\/64805"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/media\/61577"}],"wp:attachment":[{"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/media?parent=61458"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/categories?post=61458"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.altoros.com\/blog\/wp-json\/wp\/v2\/tags?post=61458"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}