We at Monkton use DynamoDB a lot for storage. It is extremely fast and scalable. A lot of the work we do is in AWS GovCloud, so this post will be geared towards that, but easily portable to other regions. We spent some time digging around and being frustrated trying to get this to work and wanted to share lessons learned to avoid those headaches.
Defining the need
We are helping build a new set of services, part of our multi-account architecture is a centralized "Identity SaaS" service. While we have micro-services available in that account to read/write to the "Identity SaaS" service DynamoDB, we opted to read/write directly to it, for other trusted services and accounts. This was simply a performance choice on our end to speed things up. We wanted to avoid creating a HTTPS request, waiting for it to do its thing in DynamoDB, and return—when we could do it directly using the same logic.
Many considerations
Part of how to configure this is understanding where and what services we will be using. For this project, we are using Lambda and ECS Fargate to deploy backend services. For the purposes of this demo, we are looking at Fargate, but lessons apply to Lambda as well. Part of that is following "Best Practices" and deploying these services into VPCs with private subnets.
Account Layout
As a starting point, as stated above we use AWS Organizations. We follow the "Root Account," Logging account, Security account breakout. Within that, for the purposes of this walk through, we also have the "Identity SaaS" app account and the "Standalone App" account.
A very simple high level overview:
As stated above, we have a "master-account" table that holds account details in the "Identity SaaS" account. We want our "Standalone App" to be able to read/write directly into that DynamoDB instance in the "Identity SaaS" account.
You will need to configure three things:
- A role within the "Identity SaaS" account that enables reading/writing to that "master-account" DynamoDB table (as well as the KMS keys protecting it, because security)
- Grant the Fargate Role the right to assume the role for "Standalone App"
- Update or create a VPC Endpoint in the "Standalone App" that enables access to the DynamoDB "master-account" in the "Identity SaaS" account
"Identity SaaS" Role
This is a snipped from the CloudFormation YAML file, that creates the role. You'll notice the parAppDynamoDBPrefix and parOrganizationIdentifier parameters, these can be modified or removed depending on your desired end state.
For demonstration purposes, the Condition for our Role can be tightened up a bit. We are allowing any account in our Organization the right to read/write. You may want to limit it to certain OU within that org or something else. Additionally, you may want to tighten up the Principal to limit to the specific service you are calling from.
Be sure to tighten the permissions to what you want other accounts to be able to do. Perhaps deleting or updating should be very limited—it depends on your use case.
# A role needed for cross account connections
AppDynamoDBConnectionRole:
Type: AWS::IAM::Role
Properties:
RoleName: "master-dynamodb-account-role"
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
AWS: "*"
Condition:
StringEquals:
aws:PrincipalOrgID:
- !Ref parOrganizationIdentifier
Action:
sts:AssumeRole
Policies:
- PolicyName: app-policy-dynamo
PolicyDocument:
Statement:
- Effect: Allow
Action:
- dynamodb:UpdateItem
- dynamodb:PutItem
- dynamodb:GetItem
- dynamodb:DeleteItem
- dynamodb:Query
- dynamodb:Scan
- dynamodb:BatchWriteItem
- dynamodb:BatchGetItem
Resource:
- !Sub 'arn:${AWS::Partition}:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${parAppDynamoDBPrefix}*'
- PolicyName: app-policy-kms
PolicyDocument:
Statement:
- Effect: Allow
Action:
- kms:Decrypt
- kms:Encrypt
- kms:GenerateDataKey
Resource:
- !GetAtt AppKMSRDSKey.Arn
Once deployed into the "Identity SaaS" Role account, this allows the caller to assume the role to interact with the service.
Grant the Fargate Role Permissions
Within your CloudFormation template that deploys your Fargate tasks ("Standalone App" Account), you will want to update the TaskRoleArn Role that you have created. The additional policy here simply allows your Fargate task to assume the role in the other account. If you are deploying to Lambda, you'd apply this same permission. You will notice the parAppSharedDataAccountId and parAppSharedDataRoleName parameters, these simply allow us to pass in the "Identity SaaS" Account ID and the role name which we created in the "Identity SaaS" account.
- PolicyName: app-cross-account-dynamodb-assume-role
PolicyDocument:
Statement:
- Effect: Allow
Action:
- sts:AssumeRole
Resource:
- !Sub 'arn:${AWS::Partition}:iam::${parAppSharedDataAccountId}:role/${parAppSharedDataRoleName}'
Once deployed to the "Standalone App" Account, this will enable your Fargate Task or Lambda function the ability to interact with the DynamoDB table, almost...
Update or create a VPC Endpoint
This should be deployed to your "Standalone App" as well. Part of this may exist for you already, it just needs to be updated.
In this snippet from our CloudFormation which sets up the VPCEndpoint to read from DynamoDB in a private subnet. You'll notice the condition HasExternalCrossDataShareAccount, which allows us to add this shared account identifier if need be. The lack of the parAppSharedDataAccountId would prevent this from being inserted.
You will notice this has been left wide open with the dynamodb:* permissions. This should be tightened up by you, but you should also be enforcing those permissions with the role and access in your Fargate Tasks as well as the role created in the "Identity SaaS" account.
AppFunctionVPCEndpointForDynamo:
Type: AWS::EC2::VPCEndpoint
Properties:
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal: "*"
Action:
- "dynamodb:*"
Resource:
- !Sub 'arn:${AWS::Partition}:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${parAppDynamoDBPrefix}*'
# External
- !If [ HasExternalCrossDataShareAccount , !Sub 'arn:${AWS::Partition}:dynamodb:${AWS::Region}:${parAppSharedDataAccountId}:table/*' , !Ref "AWS::NoValue" ]
RouteTableIds:
- !Ref AppFormationPrivateRouteTable
ServiceName: !Sub com.amazonaws.${AWS::Region}.dynamodb
VpcId: !Ref AppFormationVPC
Off to the races
Once this has been applied to the VPC, you will be able to perform all the actions you want by assuming the role in your "Standalone App" account that resides in your "Identity SaaS" account.
Happy clouding.