DigitalGlobe Builds Satellite Control System with Microservices and Cloud Foundry

by Roger StrukhoffJuly 17, 2015
The company's Cloud Foundry PaaS initiative automates testing and increases overall velocity for its product development.
Why read this?
Use Case for Cloud Foundry:

Communications: DigitalGlobe

Business or Technical Result:

Development is much faster and operational issues can be handled much more efficiently and effectively.

Lessons learned:

Transitioning from legacy to PaaS isn’t accomplished all at once. Three specific “patterns” of apps and services have emerged.

CF deployed to:

Deliver software for ground systems that communicate with latest mapping satellites.

What else is in the stack?

OpenStack

Cool fact about deployment:

Controls the ground system that controls the satellites.

Company Description:

DigitalGlobe is located in Longmont, CO, has revenues of $400+ million, and maps the world with its satellites and imaging software for Google Maps.

Cool fact about the company:

They make satellites, dude.

(Note: This use case is based on a presentation at the Cloud Foundry Summit 2015 in Santa Clara, CA by CTO and others from DigitalGlobe.)

DigitalGlobe provides satellite imagery for Google Maps. It develops its own satellites as well as the software that enables them. The company recently undertook an initiative built on Cloud Foundry as part of building a new ground system that controls its latest satellites. A couple of slides describing the company’s satellites and its latest undertaking are shown below:

DigitalGlobe adopting Cloud Foundry

digitalglobe-business-challenge-satellite

“We identified how we wanted it to work and PaaS was the way it should go,” according to company CTO Mark Carlson. “We evaluated the offerings out there and Cloud Foundry was a pretty obvious winner.”

The company downloaded both the open-source and Pivotal versions of Cloud Foundry and pressure-tested them “to make sure they lived up to how we believed they should function,” according to Carlson. An overview of the technical decision process is shown in the slide below:

DigitalGlobe Use Case: Technical decision process with Cloud Foundry

The testing and evaluation process

As part of the initial testing and evaluation process, DigitalGlobe team members ran a series of applications and services on OpenStack, Amazon Web Services (AWS), and VMware. They also employed Java, Ruby, and Python buildpacks, running apps and services on all the combinations of IaaS and language.

In doing so, “we wanted to challenge the status quo, so we moved a lot of people’s cheese,” Carlson notes. Scrum teams were created and made autonomous from one another–and from IT operations–as much as possible. With top management buy-in, they then expanded the concept and use of end-to-end testing and demonstration.

Along the way, they saw the benefits of focusing on an alignment of Scrum teams by work product, not technology. A summary of goals and processes is provided in the following table:

 

Goal
Process/Product/Technology
Rapid development with minimal dragScaled Agile Framework (SAFe)
Continuous IntegrationDevOps & Cloud Foundry
Automated Deployment PipelineJenkins
Environmental ParityCloud Foundry Buildpacks
Burstable Environment: Elasticity/Multi-Platform (Cloud Enablement)Cloud Foundry/Moab HPC
Microservice EnablementWell-Defined Interfaces/Pragmatic Architecture

 

Carlson points out that the aim with the burstable environment “is the ability to have code on- and off-premises and be able to move it back and forth.” Microservice enablement was of course a key goal, but DigitalGlobe also found it needed to be pragmatic with the overall architecture.

An overview of technical accomplishments so far is provided in the following slide:

DigitalGlobe Use Case: Technical accomplishments with Cloud Foundry

Three patterns emerge

Along the way, the company defined three architectural patterns. Ideally, all apps and services would fit into what DigitalGlobe calls Pattern 1, a top layer that fits well into a PaaS and conforms to a microservices architecture. But there were cases where legacy apps needed to be put into VMs instead, and formed what DigitalGlobe calls Pattern 2 applications. A third layer of Pattern 3 apps are those “where we need every millisecond of compute power, and the latency that a virtual environment causes is unacceptable,” according to DigitalGlobe’s Steve Wall. “The company needs a high-performance, bare-metal compute cluster in this case.”

“It’s not an all or nothing proposition,” Wall says. “With all of the apps and services, it’s a question of what fits well where?”

An overview of the three patterns is shown in the following slide:

DigitalGlobe Use Case: Cloud Foundry architechture app patterns

Cloud Foundry Professional Services

To go where none have gone before

Culturally, “our path to adoption is taking the organization to a place it has never been before,” Wall points out. The process involved in doing so encompasses three key points:

  • Develop one service. The company started small, with an eventing service. The service publishes an event and distributes it out.
  • Use a small team of developers. They are known as “the pioneers.” After the first service, they developed a few more, “we got a BPM layer in to orchestrate workflow,” Wall says, “ and we had an end-to-end demo.”
  • Do the Initial buildout and then roll it out to the company at large.

Using an agile development approach “got us into understanding what a microservice was,” according to Wall. “We also learned in this case that we had three microservices so now we had a template. We got it into Git and into Jenkins, and got our deployment pipeline started.”

The company is now doing a blue/green deployment and executing some automated tests.

In fact, “automated testing is bedrock of everything we’re doing,” Wall says. “We can’t do this with manual testing because we won’t get velocity through the pipeline.”

A move to self-provisioning (see slide below) has made “every happier,” Wall says. “In the legacy environment you had to coordinate among a lot organizations.If you wanted an environment build out, you had to get a lot of people involved, and the coordination among organizations was time-consuming.”

By contrast, “in a continuous delivery environment there are now clear lines of delineation between the organizations,” he says. “We gave the application development team the power to do what they needed to do. They have access to a pool of resources, so are now able to test instances in an afternoon instead of a week or two, or conduct scale-handling tests that simply were not possible before.”

DigitalGlobe Use Case: Move to Self-provisioning

Related videos

Here is the original video presented at the Cloud Foundry Summit 2015 in Santa Clara, CA.

 

Related slides

An overview of DigitalGlobe can be found on SlideShare:

 
Interested in how to manage secure Cloud Foundry deployments distributed across multiple data centers?

To stay tuned with the latest updates, subscribe to our blog or follow @altoros.

  •  
  •  
  •  
958