Platform approach 101: Getting cloud right, the first time
Mastering cloud adoption with automation and control.
This blog, part of the Cloud Strategy Unboxed series, looks at the technology behind successful cloud adoption. A key principle of the cloud is to automate everything that can be automated. Manual checklists and organizational policies are automated in controls and pre-approved infrastructure templates.
Cloud platform 101
A workload or platform approach to managing and operating a cloud environment is a decision central to every cloud adoption initiative.
In figure 1, the path on the left focuses on shifting individual workloads into the cloud on an application-by-application basis. Typically, this will be driven by the individual business units responsible for the applications. Each team will take an approach that makes sense for their application and with limited centralization, automation, or use of shared resources. This results in deployment and operational processes specifically tailored to that particular application.
A workload approach is attractive and a common first strategy for organizations because it can deliver quick results and may initially seem like the easier option. There are few blockers to migrating workloads into the cloud and reaping the benefits as any processes and policies are adapted to suit the application, often in a fresh separate cloud account for each workload.
Challenges arise as more and more applications are migrated or developed, and the organization finds that the processes established with earlier applications are not well suited to each following one. Exemptions and changes spiral out of control. Each application has its own unique set of processes, or centralized processes are full of exceptions and patches to make them work for all applications. Either way, migrations seem to become more complex, mired in bottlenecks and an increasing backlog of tickets. Operations teams struggle to maintain a healthy level of availability and support, as each application requires a different approach for monitoring, patching and maintenance. The inconsistent processes inevitably lead to security holes and auditing issues.
For organizations trying to evolve to more modern architectures, the challenges and delays will be even more pronounced. The increasing number of infrastructure components in such architectures leads to an explosion of support tickets for provisioning, changing, and maintaining the cloud resources.
The path on the right uses a central cloud platform that provides a consistent control plane across all cloud applications within the organization. This approach is intended to bring long-term benefits. The challenge is that considerable short-term effort is needed to create the team and build the platform before applications can be migrated and the benefits of the cloud are reaped.
A platform approach forces all application teams to deploy their applications the same way and comply with a consistent set of policies, with limited tolerance for unnecessary exceptions. This is often seen as a blocker and means teams need education and support for this approach to succeed.
These challenges are precisely why the Cloud Centre of Excellence should include the Education and Enablement teams. No strategy stands alone; organizational cloud adoption consists of multiple strategies, all working towards the ideal state. The education team should focus on communicating the benefits of a platform approach. For example, while application teams are limited to consistent policies and are expected to provision their own infrastructure, this approach enables them to deploy faster, more frequently, and with less waiting on manual processes and stage gates.
It’s never too late for an organization to change its path. Prior experience with a workload approach helps an organization understand the value of a platform approach and justifies the initial effort required to take this path. Time spent in the cloud will also increase the experience and cloud fluency of the organization, making it easier for teams to use a cloud platform. For example, “ClickOps” is a common approach to managing cloud infrastructure for organizations new to the cloud. In this approach, infrastructure is manually created via the cloud service provider’s user interface. While it has a bad reputation (for many reasons), going from ClickOps to Infrastructure-as-Code is easier than going from exclusively on-premises infrastructure to infrastructure-as-code in the cloud.
Controls
Controls are an essential part of enabling developers. They automatically enforce compliance and other policies in the cloud account or deployment pipeline. These controls, combined with pre-approved configurable infrastructure templates, mean that there is no longer a need for separate teams to provision and own infrastructure in the cloud. Instead, in a modern cloud environment, application teams provision infrastructure themselves as part of the application development process.
The controls enforce restrictions on which services developers can provision and how they are configured. This is to ensure security, best practices and any regulatory requirements are complied with.
There are four types of controls.
Directive controls
First, directive controls list all the requirements and expectations for cloud usage, usually in documentation and training materials. They don’t enforce anything, only inform. Establishing the directives is an essential first step between existing security policies and cloud controls.
Preventive controls
Preventive controls are the primary means of enforcing the directives. They restrict the actions that users are allowed to take before they take them. Most organizations would be familiar with this through access permissions and allow listing.
Detective controls
Detective controls define a set of rules and actively monitor changes within the cloud environment. The controls can send notifications for any non-compliance events. Any follow-up would be at the discretion of those receiving the notifications. They are used as a secondary compliance monitoring layer for cases that can’t be prevented and for cases where a suitably authorized user might circumvent the preventive controls – also known as Shadow IT.
Corrective controls
Lastly, corrective controls are similar to detective controls but go a step further to automatically take remediation action, such as removing or rolling back the non-compliant changes. These are less commonly used as organizations are often uncomfortable with automated changes to an environment and such changes can cause issues with CI/CD pipelines. However, they are popular for data use cases. For example, a corrective control can monitor an uploads folder for any personal data that should not be there, then automatically delete it or move it to a secure location for further analysis. There are also good security use cases, such as automatically disabling access credentials when they are flagged as compromized.
Which control type to prioritize?
For productive modern cloud development, application teams need to be able to innovate in the development environment, so directive and detective controls are ideal. This is especially the case for sandbox and training accounts.
Production environments are often stricter, with preventive controls preferred.
organizations that are highly regulated, more risk-averse or with less mature application teams tend to favour a preventive and corrective approach, even in development environments. However, it’s important to note that even with this stricter approach, developers are still enabled to provision at least the approved resources themselves, making it still preferable over any approach with separate provisioning teams.
Pre-approved infrastructure templates
Modern application teams use Infrastructure as Code (IaC) templates to provision the cloud resources they need. The CCoE can create pre-approved infrastructure templates, sometimes called blueprints, for application teams to use. These templates can be for a single cloud service or for an entire application architecture, such as a 3-tier application.
Pre-approved templates serve two key goals.
- The templates are built according to general and organizational best practices and the directives. The CCoE owns the templates and can control exactly which configuration settings can be changed by the app team. For example, in a 3-tier template, they may be permitted to select an appropriate server type for their application. However, they will be unable to change the network or firewall settings.
- Templates provide a ready-to-use, high-quality architecture for application teams to easily configure and provision infrastructure. Taking away much of the trial and error and uncertainty in following best practices. Given that most enterprize applications are quite similar, this can greatly increase productivity, especially for mass migrations of applications to the cloud.
Pre-approved templates solve several other common challenges, such as inconsistent approaches to the same architecture with varying levels of quality and security. They also make application architectures more predictable, improving productivity and project cost and timeline estimates.
The CCoE should develop templates for all the common patterns that an organization needs. Typically, this is done as each first type of application is migrated or built. The result from that first mover can then be reused by following projects.
While application teams should use an existing template, those with more unique needs can use the cloud service templates to construct a custom architecture. Justified cases that can’t be implemented with any of the available templates can be addressed by working with the CCoE’s application enablement team on a special blueprint just for that particular application.
The modern cloud deployment pipeline
Creating resources via the cloud console, also known as ClickOps, is not permitted in a cloud platform strategy. The pipeline holds the deployment permissions to the cloud; application teams do not have access to the cloud console or only minimal read access to the relevant application logs. Application teams must send their IaC templates and code to the cloud via a deployment pipeline. This is also desirable for the CCoE and the organization as it provides an auditable proof of compliance.
Consider this deployment pipeline a type of pipe through which digital information can travel. Traditionally, this is all that pipelines did; they carried code from point A to point B. However, in modern cloud, they have evolved in function and importance. Modern pipelines include the ability to enforce our controls. Think of it as a series of sensors and valves throughout the length of the pipe.
If a sensor detects something that is not compliant, such as an unapproved cloud service being requested, then the pipeline automatically closes the valve, and the deployment fails before any resources have been created in the cloud. The application team is notified of the failure so they can adjust their template and try again.
This means that the pipeline is a critical part of the security in many modern cloud management strategies. If an application makes it through the pipeline without being rejected, it can be considered compliant with the enforced directives.
Pipelines also have logging capabilities. As each deployment passes through the pipeline, the enforced directives will log their evaluation of it, providing an auditable trail of compliance proof.
Self-service portal
The last part of a cloud platform that this article will address is a self-service portal. Core tenants of modern cloud strategy are to automate as much as possible and enable application teams to be self-sufficient with the rest.
On a high level, a self-service portal is an interface through which a product owner or similar business role can register new projects – also called project onboarding. Once the project is registered, the portal can automatically create the code repo, attach the deployment pipeline, and set up the application’s environments.
However, a self-service portal can potentially do a lot more. For example, as part of project onboarding, the business owner can be asked to note the data sets that the application will use so that a catalogue of sensitive data can be automatically maintained. The business owner can also be asked to indicate the expected number of users or similar, which can then be used to automatically calculate an operational cost estimate for the project. Information such as project name, team ID, and billing code can be requested. The values can be automatically included in tags attached to each provisioned cloud resource to support the organization’s FinOps strategy.
After onboarding, the business owner can manage access, assigning developers to the project who can access the created code repo and deploy their infrastructure templates and code. The portal is also a great place to build dashboards that provide status updates, logs, and alerts. For example, it can show the status of recent deployments, if they were approved or rejected by the pipeline and any messages that may be available for clarification.
Conclusion
Successful cloud adoption requires a platform approach that enables a left-shift of responsibilities to the application teams and provides a consistent control plane across all cloud applications within the organization. Although this approach requires considerable short-term effort to create the team and build the platform, it brings long-term benefits in cost, security, operations, and other areas. A modern cloud deployment pipeline, which enforces and logs compliance through controls, is critical to achieving developer enablement. Lastly, a self-service portal enables application teams to be self-sufficient, automating project onboarding, maintaining data catalogues, and supporting FinOps strategies. By adopting these technologies and strategies, organizations can maximize the benefits of cloud adoption and improve their overall productivity, project cost, and timeline estimates.
Don't miss out on unlocking the full power of the cloud. for a personalized consultation or to learn more.
