CNC: The open-source Infrastructure-as-Code Framework

Recently we announced the launch of Coherence 2.0, powered by the first Infrastructure-as-Code Framework, CNC. In this post we’ll go deeper into explaining with CNC is and what it solves for developers. 

How did we get here?

Over the last ten years Infrastructure-as-Code (IaC) has become a foundation of the DevOps movement - a way to define in code the resources that your application needs in order to run in the cloud. That said, managing IaC and using it to deploy different environments has a few key challenges:

• How to make sure IaC conforms to best practices
• How to create multiple environments from one IaC configuration
• How to manage execution between different users
• When to run IaC and where to run it: does it run in CI/CD, from dev’s laptops, or somewhere else? Does it run before every deploy, or only when needed?

As a result a category of tools we’re calling “platform orchestrator” emerged, which abstracted away the IaC. The challenge with the orchestrator is that it adds a layer of centralization that might have ok failure modes but ultimately becomes a bottleneck to creativity as applications scale.

Instead of fully abstracting IaC, we want to give developers and platform teams a way to write better IaC. That’s what the framework enables. Platform orchestrator is like going from app code to no-code. Framework is like going from writing your own webserver to using Rails or next.js. 

Our vision for CNC is an IaC templating engine that is configurable, customizable, open source, with a developer native experience.

• CNC adds a layer above IaC that renders templates using context from service definition files, and can iterate over subsets of resources for environments, enabling a native way to share some resources across multiple environments while partitioning others.some text
  • For example, you might want to use the same VPC and load balancer for multiple test environments, while having different containers and databases for each deployment
• CNC is configured using docker-compose, a familiar and developer friendly syntax that is more approachable and less error-prone than editing IaC directlysome text
  • CNC translates docker-compose into read-to-run IaC code, managing all the complexity of variables and iterations to keep guardrails in place as defined by the IaC experts on the team or in the community
• CNC has built-in reference architecture templates for common deployments, e.g. ECS on AWS or Cloud Run on GCP, but offers unlimited flexibility to customize parts of these templates or to write your own templates from scratchsome text
  • As CNC grows, more built-in reference architectures will be availablesome text
    • Examples could be:some text
      • “GPU deployments to AWS sagemaker”
      • “free services only on GCP”
• CNC is a low-level tool that can run anywhere from your own laptop to a larger internal developer platformsome text
  • There are no agents to install in the cloud, and no integrations are required. It’s just a python CLI that knows how to add the right context and render/execute the right templates, relying on cloud or IaC CLIs to do the authentication and other required integration
  • If you have some code locally and you add a docker-compose file and some environments, you can use the project fully self-contained on your own system
  • You can also run the CLI in CI/CD or integrated into tools like Backstage or Jenkins if you wanted to
• For AI apps, where access to large volumes of data for training and context has been critical, on-prem deployments have been really important so far. We’re building CNC in a way that will make it useful for defining a reference architecture for how your app should be deployed, making it easier for someone else to deploy your app into their own cloud

CNC is open source and GPL licensed, you can use it freely. With the right deployment reference architectures built in, it will be useful for even small hobby projects. And the free availability of the core components with no limits on use means that larger teams can integrate it into their more complex toolchains without limits. We know this will only happen if the framework provides value, but we’d love to see a community emerge and offer choices of different deployment variations as well as providers. Because you can see exactly how it all works, you can also always customize it for just your team too, using simple override files in your own repo. Coherence is going to support this effort as much as we can. 

Core Lifecycle Events Managed by cnc:

‍

Provision: Uses terraform to create, manage, and dismantle cloud resources, ensuring each environment is crafted to fit its specific purpose.

Build: Assembles the necessary deployment artifacts for each environment, from docker containers to static assets for web applications.

Deploy: Seamlessly updates infrastructure to deploy new artifacts, such as modifying k8s manifests or updating ECS services.

Toolbox: A toolbox is a managed shell against a cnc-managed environment, making it easy to get a REPL or run database migrations, for example

Getting Started with cnc:

Experience cnc in just a few minutes: install and see for yourself the power of cnc in under 5 minutes without needing any cloud permissions or incurring any costs.

Rapid Deployment: Have your first environment up and running in less than 15 minutes, demonstrating the straightforward power of cnc.

cnc is designed to be a powertool that empowers you to build and manage your infrastructure with the same attention to detail and creativity that you bring to your code. Just like web devs use frameworks to build better products, with cnc, you gain the freedom to implement your vision precisely as intended, making each project not only functional but finely tuned to your standards.

Hello World

‍

Install CNC

Intall cnc from the PyPI Python Package Index. For example, using pip:

pip install cocnc

Save config files

You can make a new directory, nothing but the following 2 files is needed by cnc. Save as cnc.yml:

services:
  app:
    command: "my command"
    x-cnc:
      type: backend
    build:
      context: .
  db:
    x-cnc:
      type: database
      version: 15
    image: postgres

Save as environments.yml:

name: my-first-app
provider: gcp
flavor: run-lite
version: 1

collections:
- name: dev
  region: us-east1
  base_domain: mydevsite.com
  account_id: "foo-bar-123"
  environments:
  - name: dev
    environment_variables:
    - name: FOO
      value: bar

cnc has robust environment configuration options, including support for environment variables from cloud secrets, terraform outputs, or aliasing from other variables. Read more about configuration here.

See CNC in action

All this will do is manipulate text files in your /tmp directory and won't actually touch anything in your code or cloud. It's the best wayt to get to Aha! quickly before diving in deeper.

# print the terraform we would run, as generated from the 2 ymls above
# we do --no-cleanup here so you can inspect the files yourself in /tmp if you want to.
# You can leave this off to cleanup after the command runs automatically
cnc provision debug --no-cleanup

# look at the generated build script
# you can look at the files referenced, for example the build-functions scripts, by going to the /tmp path in your terminal
cnc build perform dev --debug --no-cleanup

# same for deploy
cnc deploy perform dev --debug --no-cleanup

Add a 2nd environment (e.g. dev2) to the environments.yml and run the commands again, see the power of the framework!

Documentation

Access full documentation and in-depth tutorials here.

Community & Support

- Issues: Report bugs or suggest features via GitHub Issues.

- Support: For direct support, contact our team at cnc@withcoherence.com.

Contributing

Interested in contributing to CNC? Check out our Developers Getting Started guide for guidelines and project setup instructions.