An implementation of a code deploy system based on the answer to AlgoExpert's system design questions.
Run the get_started.sh script.
To use the aws cli tool some configuration needs to be done or the awslocalstack wrapper can be used. I prefer to use the regular aws cli as it's a straight forward configuration and using the wrappy requires downgrading to aws cli v1
This checks the health of the localstack docker container and will output what's running
curl http://localhost:4566/_localstack/health
Validate Step Functions definition
- Using file:// so the CLI reads the JSON from disk (required):
aws stepfunctions validate-state-machine-definition \
--region us-east-1 \
--endpoint-url http://localhost:4566 \
--definition file://terraform/infra/step_function_definition.json- Alternatively, cat the file into the flag:
aws stepfunctions validate-state-machine-definition \
--region us-east-1 \
--endpoint-url http://localhost:4566 \
--definition "$(cat terraform/infra/step_function_definition.json)"If you pass a plain path (e.g., ./terraform/infra/step_function_definition.json) without file:// or $(cat ...), the CLI will try to parse the literal string and fail with an INVALID_JSON_DESCRIPTION error.
- 1) Start stack and provision infra
./get-started.sh- 2) Health check
curl http://localhost:4566/_localstack/health- 3) Create a test artifact and upload to trigger the flow
printf 'hello' > /tmp/test-file.zip
aws --region us-east-1 --endpoint-url http://localhost:4566 s3 cp /tmp/test-file.zip s3://global-builds/test-file.zip- 4) Verify Step Function execution started
STEP_FUNCTION_ARN=$(terraform -chdir=/Users/nousunio/Repos/Learnings/algoexpert/system-design-practicals/iq1-code-deployment/terraform/infra output -raw step_function_arn)
aws --region us-east-1 --endpoint-url http://localhost:4566 stepfunctions list-executions \
--state-machine-arn "$STEP_FUNCTION_ARN"
EXEC_ARN=$(aws --region us-east-1 --endpoint-url http://localhost:4566 stepfunctions list-executions \
--state-machine-arn "$STEP_FUNCTION_ARN" --query 'executions[0].executionArn' -r)
aws --region us-east-1 --endpoint-url http://localhost:4566 stepfunctions describe-execution \
--execution-arn "$EXEC_ARN" | cat- 5) Verify objects copied to destination buckets
aws --region us-east-1 --endpoint-url http://localhost:4566 s3 ls s3://region-a-builds/
aws --region us-east-1 --endpoint-url http://localhost:4566 s3 ls s3://region-b-builds/- 6) Verify tracking record in DynamoDB
aws --region us-east-1 --endpoint-url http://localhost:4566 dynamodb scan \
--table-name FileCopyTracking | cat- Direct Step Function invocation
aws --region us-east-1 --endpoint-url http://localhost:4566 stepfunctions start-execution \
--state-machine-arn "$STEP_FUNCTION_ARN" \
--input '{"sourceBucket":"global-builds","sourceKey":"test-file.zip","destinationBuckets":["region-a-builds","region-b-builds"]}'- Invoke Lambda invoker directly
aws --region us-east-1 --endpoint-url http://localhost:4566 lambda invoke \
--function-name step-function-invoker \
--payload '{"sourceBucket":"global-builds","sourceKey":"test-file.zip","destinationBuckets":["region-a-builds","region-b-builds"]}' \
/tmp/invoker_out.json | cat
cat /tmp/invoker_out.json- Optional: invoke via API Gateway
REST_ID=$(aws --region us-east-1 --endpoint-url http://localhost:4566 apigateway get-rest-apis \
--query 'items[?name==`code-deploy-api`].id' -r)
# Trigger replication worker via POST /deploy
curl -s -X POST "http://localhost:4566/restapis/${REST_ID}/dev/_user_request_/deploy" -d '{}' | cat- Use
aws --endpoint-url http://localhost:4566 ...for every AWS CLI call. - To validate the Step Function definition:
aws --region us-east-1 --endpoint-url http://localhost:4566 stepfunctions validate-state-machine-definition \
--definition file:///Users/nousunio/Repos/Learnings/algoexpert/system-design-practicals/iq1-code-deployment/terraform/infra/step_function_definition.json- If something wedges, bring the stack down and re-run:
cd /Users/nousunio/Repos/Learnings/algoexpert/system-design-practicals/iq1-code-deployment && docker compose down -v
bash get-started.sh- You can also tail LocalStack logs to see Lambda/StepFunctions activity:
docker logs -f localstack-main | cat-
After a success, you should see:
- A Step Functions execution that finishes successfully.
test-file.zipinregion-a-builds/andregion-b-builds/.- A record in
FileCopyTracking.
-
If you want, I can run these for you from here.
bash /Users/nousunio/Repos/Learnings/algoexpert/system-design-practicals/iq1-code-deployment/get-started.sh
curl http://localhost:4566/_localstack/health
printf 'hello' > /tmp/test-file.zip
aws --region us-east-1 --endpoint-url http://localhost:4566 s3 cp /tmp/test-file.zip s3://global-builds/test-file.zip
STEP_FUNCTION_ARN=$(terraform -chdir=/Users/nousunio/Repos/Learnings/algoexpert/system-design-practicals/iq1-code-deployment/terraform/infra output -raw step_function_arn)
aws --region us-east-1 --endpoint-url http://localhost:4566 stepfunctions list-executions \
--state-machine-arn "$STEP_FUNCTION_ARN"
EXEC_ARN=$(aws --region us-east-1 --endpoint-url http://localhost:4566 stepfunctions list-executions \
--state-machine-arn "$STEP_FUNCTION_ARN" --query 'executions[0].executionArn' -r)
aws --region us-east-1 --endpoint-url http://localhost:4566 stepfunctions describe-execution \
--execution-arn "$EXEC_ARN" | cat
aws --region us-east-1 --endpoint-url http://localhost:4566 s3 ls s3://region-a-builds/
aws --region us-east-1 --endpoint-url http://localhost:4566 s3 ls s3://region-b-builds/
aws --region us-east-1 --endpoint-url http://localhost:4566 dynamodb scan \
--table-name FileCopyTracking | cat
aws --region us-east-1 --endpoint-url http://localhost:4566 stepfunctions start-execution \
--state-machine-arn "$STEP_FUNCTION_ARN" \
--input '{"sourceBucket":"global-builds","sourceKey":"test-file.zip","destinationBuckets":["region-a-builds","region-b-builds"]}'
aws --region us-east-1 --endpoint-url http://localhost:4566 lambda invoke \
--function-name step-function-invoker \
--payload '{"sourceBucket":"global-builds","sourceKey":"test-file.zip","destinationBuckets":["region-a-builds","region-b-builds"]}' \
/tmp/invoker_out.json | cat
cat /tmp/invoker_out.json
REST_ID=$(aws --region us-east-1 --endpoint-url http://localhost:4566 apigateway get-rest-apis \
--query 'items[?name==`code-deploy-api`].id' -r)
# Trigger replication worker via POST /deploy
curl -s -X POST "http://localhost:4566/restapis/${REST_ID}/dev/_user_request_/deploy" -d '{}' | cat
aws --region us-east-1 --endpoint-url http://localhost:4566 stepfunctions validate-state-machine-definition \
--definition file:///Users/nousunio/Repos/Learnings/algoexpert/system-design-practicals/iq1-code-deployment/terraform/infra/step_function_definition.json
cd /Users/nousunio/Repos/Learnings/algoexpert/system-design-practicals/iq1-code-deployment && docker compose down -v
bash get-started.sh
docker logs -f localstack-main | cat