Removing orphan records from Azure App Configuration Service when importing the data

Of late we have been working on putting our configuration data on App Configuration Service (ACS). We deploy all static configuration data to ACS using azure pipelines by defining all the configuration data in json files in repos.

Below is the sample json.

And storing the data in ACS as below ( other supported separators are ‘.’, ‘,’, ‘;’, ‘-’, ‘_’, ‘__’, ‘/’, ‘:’)

    Key                 value-----------------       -------service1:op1:node1    sample dataservice1:op2:node2    testing data

roblem: We ran into issues related to orphan records in ACS. Scenario is as below.

For example, when a team no longer supports op2(above example) and removes data from json file. After deploying the json file to ACS, expectation is that op2 will be deleted from ACS too. But the data is not getting deleted from ACS and the data resides as orphan data until someone deletes it explicitly using.

az appconfig kv delete

But the data in ACS will remain same.

Key                 value-----------------       -------service1:op1:node1    sample dataservice1:op2:node2    testing data

olution: Workaround to address the orphan records issue in ACS

  1. Read each json file and insert into ACS.
  2. Construct the key from the json file (e.g. service1:op1:node1).
  3. Compare the keys in ACS and the keys that are constructed from json file (in step 2).
  4. If there are any delta’s in ACS, delete those keys from ACS.

Read each json file and insert data to ACS

Get-ChildItem -Path "." -Recurse -Filter *.json |ForEach-Object {az appconfig kv import --name nameoftheacs -s file --format json --path $_.FullName --auth-mode key --connection-string $connString --depth 3 --separator : --yes

Above script will loop through each file that needs to be deployed to ACS, and will insert the data to ACS.

— auth-mode: can be key or login.

— depth: 3 (key will be constructed like service1:op1:node1 in ACS)

if depth: 2 (key will be constructed like service1:op1)

Construct keys from json

Below powershell script is used to construct the 3 node key (as shown in the example service1:op1:node1)

Compare keys from ACS with keys constructed above

Read keys from ACS using CLI.

$keys = az appconfig kv list --key "*" --label {label name to query} --fields key --output tsv  //pass other parameters as needed

Compare and delete the orphan keys from ACS

Run the complete powershell in pipeline task. It will perform CRUD on ACS data.

Conclusion: To our surprise import cli command doesn’t perform the deletion of orphan records, and raised an enhancement request here. Until then, we have a workaround to perform CRUD on ACS data. Please do let me know if you have implemented any similar solution in a better way.

References:

--

--

Azure Full Stack Engineer @Ford Motor company

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store