Imagine that you want to automate the end-to-end process of:
- Provisioning DSC Pull server with SSL support.
- Registering DSC clients per their infrastructure role.
- Deploying a DSC configuration to all of the clients – or specific client role, once you commit your changes to the repository.
This blog post is covering all components and processes used to automate above end-to-end process.
I assume that you have a strong understanding of the DSC architecture and design, otherwise, it might be difficult to follow and understand the idea of this process.
- The explained process was applied to one of the client infrastructures, all confidential information has been substituted with the name of the custom.company , in order to preserve privacy.
- All code used in this post is stored in the form of Powershell module and can be found at the link: AutomatingDSC
First, I will wrap up the scenario of the environment and all of the components that are used in the process, in order to piece them together later.
- Infrastructure is being hosted on AWS cloud.
- Code is being stored in GitLab(Git) repository, we are leveraging pipelines.
- Cloudformation is being used to provision the EC2 instance, in the CF YAML file, we are using Powershell for initial bootstrap process.
- We are leveraging Nuget server to store internal Powershell module for the EC2 bootstrap.
- All objects are being stored in the S3 bucket.
- All secrets are being stored in the Systems Manager Parameter Store.
- Tags are being leveraged to store key=pair value of the infrastructure role(DSC configuration) and the unique ID of each client.
- In the upcoming text, I will break down the whole process in-depth, step-by-step, with the code snippets and comments for each line of code.
- Additionally – every function in the module is described with comments.
- Below is the diagram, which is describing roughly components that are used.
- High-Level Overview: Idea which leverages this automation is quite simple or complex, we spin up the DSC pull server which will deliver configuration to various server types in our infrastructure. Server types can be like, Active Directory Master, Active Directory Slave and so on. All configuration data for the server types is stored in the GitLab(Git) repo, whenever we commit some new changes to the configuration code, it is being copied over to the DSC server via pipeline logic and new MOF files are generated. Probably you are wondering how are the MOF files generated? Well, once the new server is being spined up, like AD Master, it registers itself with the DSC server with the unique ID, then it stores that ID as the AWS EC2 tag in the format of ADMaster=UniqueID. Later on, DSC server will do the query for all of the tags that contain in the name ADMaster and get the IDs, then it will generate new MOFs with the configuration dedicated to that role. This gives us lots of flexibility, does not matter how many servers we add under the same role name, DSC server will on schedule always recognize them by searching for the EC2 tag and generate the configuration.
- Layout – of the explanation, if you click on any function(bolded) name, it will redirect you to the GitHub page where code is posted. If the function contains the helper(sub-function) in its code, it will be marked with an under-dot sign.
- In the beginning, we have deployed our EC2 template with the CloudFormation, in the CF YAML file, there is a Powershell section, which is being used to bootstrap initially the instance. Bootstrap process is quite basic, we register new Powershell repository(Internal Nuget Feed Server – setup is not covered in this blog post) and download certain modules from there, after that we import the main module and execute the commands/functions from it. The code snippet below.
# Distinguished name of the repository that we are going to register - Internal Nuget feed $Repository = 'nuget.custom.company.aws.com' # Register Powershell repository Register-PSRepository -Name $Repository -SourceLocation "http://$($Repository):8443/nuget" # Install Custom Company dedicated Powershell module Install-Module -Name 'CC' -Repository $Repository -Scope AllUsers # Import CC module and AWSPowershell Module Import-Module CC,AWSPowershell # Update IP address of the DSC server in the Route53 DNS, so all internal servers can reach it Edit-R53IPAddress -Name 'dsc.custom.company.aws.com' -ZoneID 218hsa1232 -Comment 'Editing DSC server IP address' # Main function for installing the DSC server Install-DSCServer # Function for installing and configuring the Powershell Core(6) with the SSH support # This is necessary because GitLab runner is being hosted on Docker(Linux) container which is running Powershell Core version # There are several issues with the classing Powershell remoting in the Core version, that's why we use SSH support in Powershell Install-PSCore
- Let’s really break down what happened in the code above. We registered the internal repository, downloaded the module from it which contains all necessary functions which have been executed in the specific sequence.
- Edit-R53IPAddress , Route53 DNS has been updated with the recent IP address of the DSC server, remember – idempotency, if our instance is terminated, we can just spin up the new one from the code.
- Install-DSCServer , in the background, the system installs required modules that are being used in the configuration block, then configuration block is coming, SSL self-signed certificate is generated with the expiry date of 100 years from now, Windows Feature required for the pull server is installed, web service is configured, registration key – which will be used later on from the client-side(to register them) is generated, firewall rule is added to allow inbound traffic to the DSC port and folder where configuration scripts will be stored is created.
- Copy-Certificate , this helper function is copying the self-signed certificate object to the trusted root store of the DSC server itself, better sure than sorry.
- SSL self-signed certificate is being stored at the S3 bucket, so clients can retrieve it and store in the trusted root store(we are not using internal certification authority).
- Registration Key is being stored in the SSM parameter store, so clients, later on, can retrieve it and use it to register themselves against DSC server.
New-DSCEncryptionCertificate , this function is generating self-signed certificate with the validity of 100 years from now, dedicated to encryption of the credentials in MOF files, it stores the PFX type of the certificate in the S3 bucket and writes decryption key in the SSM parameter store, so the clients are able later-on to import the same certificate, trust it and use to decrypt the credentials from the MOF files. The function is based on official Microsoft guide – LINK
- New-CCAdmin , this function creates the local user account and makes it a member of the administrator group. Password is pre-stored in the SSM parameter store. Purpose of this is creating the configuration publishing schedule job – later-on.
Register-DSCSchedule , function which registers DSC configuration publishing task, using the above-created account from the previously executed function.
- Pipeline is being triggered, so if there is any code already in the GIT repository, it will be deployed to the DSC server.
- Install-PSCore , function which is installing the Powershell Core version, along with enabling the SSH functionality of the server.
- At this point, we have fully functional DSC Pull server, which already published configuration for specific infrastructure roles and has a scheduled task ready, so if there are any new clients being registered in the future, it will recognize them by using Publish-DSCCustomConfiguration function.
Publish-DSCCustomConfiguration , this function is very powerful(took me some time to build it), in the essence it is used to generate the MOF files for the all or specific infrastructure roles and deploy that to DSC configuration path.Configuration example. This function contains as well Publish-DSCModule helper function.
- Publish-DSCModule , a helper function is used to install and publish all dependency DSC modules which are used in the configuration files.
All right! We went over the process of spinning up the DSC pull server, how it generates the configuration for the various infrastructure roles on the schedule and via pipeline. You can check the pipeline code here –
Its now time to explain the registration of the DSC client, once you understand this piece, you will also better understand the logic of the DSC configuration publishing.
- Register-DSCClient , this function is registering client against the DSC pull server, also generate unique ID which is saved in the TAG of EC2 instance. This value is used by the DSC server while generating the MOF file for the specific infrastructure role.
Nowadays a strong focus on automation is crucial, I have just explained how I have built logic to automate the automation.