AWS DynamoDB实战之表的设计

2023-12-30 13:37:58

案例

使用dynamodb实现一个简单的收藏功能,参考微信的收藏功能,功能点包括以下:

对多种类型的数据进行收藏,包括图片视频,链接,文件,音乐,聊天记录,语音,笔记,位置等
对收藏的数据进行取消收藏
查看所有类型的收藏数据,按收藏时间降序进行排序
查看特定类型的收藏数据,按收藏时间降序进行排序
创建标签,删除标签,编辑标签名称
展示所有的标签
对收藏的数据进行标签管理
查看某个标签下所有类型的收藏数据,按收藏时间降序进行排序
查看某个标签下特定类型的收藏数据,按收藏时间降序进行排序
对所有收藏数据进行搜索

表设计

针对上述案例,初步来看,需要三张表:

TagDto,用于记录用户创建的所有标签
FavoriteDataDto,用于记录用户收藏的数据的基本信息
FavoriteDataTagDto,用于记录收藏数据的tag信息

根据aws dynamodb的最佳实践: As a general rule, you should maintain as few tables as possible in a DynamoDB application.

所以这里我们只需要创建一个dynamodb表(develop.Favorite), 三个dto结构共用同一张表.

TagDto

首先来看TagDto,tag的基本操作包括创建,删除,编辑名称,从这几个操作分析,TagDto只需要包含useId,tagId,tagName这三个字段就够了.

另外从展示所有tag角度出发,还需要一个排序操作,排序可以有多种方式,比如:

按照tag创建的时间进行排序
按照tag的名称进行排序
按照tag的最近访问时间进行排序

为了支持多种排序方式,我们再添加crateTime, lastAccessTime两个字段.

综上,TagDto共包含以下5个字段:

userId
tagId
tagName
crateTime
lastAccessTime

定好字段之后,再来看键的设计. 根据需求,TagDto有如下特性:

一个用户可以创建多个tag
tagName可变
tagId不可变
需要获取某个user创建的所有tag

综上,我们需要以userId作为分区键, 以tagId作为排序键来创建一个组合键.

此外,还需要支持三种排序,所以需要创建三个Local Secondary Index. 对应的排序键分别为tagName, crateTime, lastAccessTime.

package com.jessica.dynamodb.favorite.dto;

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIgnore;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIndexRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable;
import com.jessica.dynamodb.constant.DynamoDBConstant;
import com.jessica.dynamodb.utils.KeyGenerator;

import lombok.Data;

@Data
@DynamoDBTable(tableName = "develop.Favorite")
public class TagDto {
	private static final String DTO_NAME = "Tag";

	@DynamoDBIgnore
	private String userId;
	@DynamoDBIgnore
	private String tagId;
	@DynamoDBAttribute
	private String tagName;
	@DynamoDBAttribute
	private long createTime;
	@DynamoDBAttribute
	private long lastAccessTime;

	@DynamoDBHashKey
	public String getPk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId);
	}

	public void setPk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
	}

	@DynamoDBRangeKey
	public String getSk() {
		return KeyGenerator.createRangeKey(tagId);
	}

	public void setSk(String rangeKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, rangeKey);
		this.tagId = keys[0];
	}

	@DynamoDBIndexRangeKey(localSecondaryIndexName = DynamoDBConstant.LSI_ONE_NAME)
	public String getLsiOneSk() {
		return KeyGenerator.createRangeKey(tagName);
	}

	@DynamoDBIndexRangeKey(localSecondaryIndexName = DynamoDBConstant.LSI_TWO_NAME)
	public String getLsiTwoSk() {
		return KeyGenerator.createRangeKey(String.valueOf(createTime));
	}

	@DynamoDBIndexRangeKey(localSecondaryIndexName = DynamoDBConstant.LSI_THREE_NAME)
	public String getLsiThreeSk() {
		return KeyGenerator.createRangeKey(String.valueOf(lastAccessTime));
	}
}

FavoriteDataDto表

参考微信的收藏数据设计,包含创建者信息creatorId,缩略图thumbnailUrl,链接contentUrl,标题title,数据类型dataType.

根据需求,FavoriteDataDto有如下特性:

获取用户收藏的所有数据,并按收藏时间降序排序
获取所有某种特定类型的收藏数据,并按收藏时间降序排序

为了支持按收藏时间进行排序,我们再添加clipTime字段.

综上,FavoriteDataDto共包含以下8个字段:

userId
dataId
creatorId
title
thumbnailUrl
contentUrl
dataType
clipTime

我们需要以userId作为分区键, 以dataId作为排序键来创建一个组合键,这样就可以支持获取用户收藏的所有数据,只需要以userId作为hashKey进行query操作即可.

此外,还需要创建一个Global Secondary Index, 分区键为userId&type, 排序键为clipTime&dataId,这样就可以支持获取所有某种特定类型的收藏数据,只需要以userId&type作为hashKey,对Global Secondary Index进行query操作即可.

package com.jessica.dynamodb.favorite.dto;

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIgnore;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIndexHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIndexRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConvertedEnum;
import com.jessica.dynamodb.constant.DynamoDBConstant;
import com.jessica.dynamodb.utils.KeyGenerator;

import lombok.Data;

@Data
@DynamoDBTable(tableName = "develop.Favorite")
public class FavoriteDataDto {
	private static final String DTO_NAME = "FavoriteData";

	@DynamoDBIgnore
	private String userId;
	@DynamoDBIgnore
	private String dataId;
	@DynamoDBAttribute
	private String creatorId;
	@DynamoDBAttribute
	private String title;
	@DynamoDBAttribute
	private String thumbnailUrl;
	@DynamoDBAttribute
	private String contentUrl;
	@DynamoDBTypeConvertedEnum
	@DynamoDBAttribute
	private FavoriteDataType dataType;
	@DynamoDBAttribute
	private String clipTime;

	@DynamoDBHashKey
	public String getPk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId);
	}

	public void setPk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
	}

	@DynamoDBRangeKey
	public String getSk() {
		return KeyGenerator.createRangeKey(dataId);
	}

	public void setSk(String rangeKey) {
		String[] keys = KeyGenerator.parseRangeKey(rangeKey);
		this.dataId = keys[0];
	}

	@DynamoDBIndexHashKey(globalSecondaryIndexName = DynamoDBConstant.GSI_ONE_NAME)
	public String getGsiOnePk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId, dataType.getValue());
	}

	public void setGsiOnePk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
		this.dataType = FavoriteDataType.valueOf(keys[1]);
	}

	@DynamoDBIndexRangeKey(globalSecondaryIndexName = DynamoDBConstant.GSI_ONE_NAME)
	public String getGsiOneSk() {
		return KeyGenerator.createRangeKey(clipTime, dataId);
	}

	public void setGsiOneSk(String rangeKey) {
		String[] keys = KeyGenerator.parseRangeKey(rangeKey);
		this.clipTime = keys[0];
		this.dataId = keys[1];
	}
}


package com.jessica.dynamodb.favorite.dto;

public enum FavoriteDataType {
	IMAGE("image"),
	LINK("link"),
	FILE("file"),
	POSITION("position");
	
	private String value;
	
	private FavoriteDataType(String strValue) {
		this.value = strValue;
	}
	
	public String getValue() {
		return value;
	}
}

FavoriteDataTagDto

FavoriteDataTagDto用于记录用户对数据添加的tag,所以需要包含userId,dataId,tagId字段.根据需求,FavoriteDataTagDto有如下特性:

获取被收藏数据上添加的所有tag
获取所有添加了某个tag的收藏数据,并按收藏时间降序排序
获取所有添加了某个tag的某种特定类型的收藏数据,并按收藏时间降序排序

根据后两个特性,该表还需要添加clipTime和dataType字段.

综上,FavoriteDataTagDto共包含以下5个字段:

userId
dataId
tagId
dataType
clipTime

我们需要以userId&dataId作为分区键, 以tagId作为排序键来创建一个组合键,这样就可以支持获取获取被收藏数据上添加的所有tag,只需要以userId&dataId作为hashKey进行query操作即可.

此外,还需要创建两个Global Secondary Index.

第一个Global Secondary Index的分区键为userId&tagId, 排序键为clipTime&dataId,这样就可以支持获取所有添加了某个tag的收藏数据,只需要以userId&tagId作为hashKey,对第一个Global Secondary Index进行query操作即可.

第二个Global Secondary Index的分区键为userId&tagId&type, 排序键为clipTime&dataId,这样就可以支持获取所有添加了某个tag的某种特定类型的收藏数据,只需要以userId&tagId&type作为hashKey,对第二个Global Secondary Index进行query操作即可.

package com.jessica.dynamodb.favorite.dto;

import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBAttribute;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIgnore;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIndexHashKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBIndexRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBRangeKey;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTable;
import com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTypeConvertedEnum;
import com.jessica.dynamodb.constant.DynamoDBConstant;
import com.jessica.dynamodb.utils.KeyGenerator;

import lombok.Data;

@Data
@DynamoDBTable(tableName = "develop.Favorite")
public class FavoriteDataTagDto {
	private static final String DTO_NAME = "FavoriteDataTag";

	@DynamoDBIgnore
	private String userId;
	@DynamoDBIgnore
	private String dataId;
	@DynamoDBIgnore
	private String tagId;
	@DynamoDBAttribute
	private String clipTime;
	@DynamoDBTypeConvertedEnum
	@DynamoDBAttribute
	private FavoriteDataType dataType;

	@DynamoDBHashKey
	public String getPk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId, dataId);
	}

	public void setPk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
		this.dataId = keys[1];
	}

	@DynamoDBRangeKey
	public String getSk() {
		return KeyGenerator.createRangeKey(tagId);
	}

	public void setSk(String rangeKey) {
		String[] keys = KeyGenerator.parseRangeKey(rangeKey);
		this.tagId = keys[0];
	}

	@DynamoDBIndexHashKey(globalSecondaryIndexName = DynamoDBConstant.GSI_ONE_NAME)
	public String getGsiOnePk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId, tagId);
	}

	public void setGsiOnePk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
		this.tagId = keys[1];
	}

	@DynamoDBIndexRangeKey(globalSecondaryIndexName = DynamoDBConstant.GSI_ONE_NAME)
	public String getGsiOneSk() {
		return KeyGenerator.createRangeKey(clipTime, dataId);
	}

	public void setGsiOneSk(String rangeKey) {
		String[] keys = KeyGenerator.parseRangeKey(rangeKey);
		this.clipTime = keys[0];
		this.dataId = keys[1];
	}

	@DynamoDBIndexHashKey(globalSecondaryIndexName = DynamoDBConstant.GSI_TWO_NAME)
	public String getGsiTwoPk() {
		return KeyGenerator.createHashKey(DTO_NAME, userId, tagId, dataType.getValue());
	}

	public void setGsiTwoPk(String hashKey) {
		String[] keys = KeyGenerator.parseHashKey(DTO_NAME, hashKey);
		this.userId = keys[0];
		this.tagId = keys[1];
		this.dataType = FavoriteDataType.valueOf(keys[2]);
	}

	@DynamoDBIndexRangeKey(globalSecondaryIndexName = DynamoDBConstant.GSI_TWO_NAME)
	public String getGsiTwoSk() {
		return KeyGenerator.createRangeKey(clipTime, dataId);
	}

	public void setGsiTwoSk(String rangeKey) {
		String[] keys = KeyGenerator.parseRangeKey(rangeKey);
		this.clipTime = keys[0];
		this.dataId = keys[1];
	}
}

工具类

package com.jessica.dynamodb.constant;

public class DynamoDBConstant {
	private DynamoDBConstant() {

	}

	public static final String HASH_KEY = "pk";
	public static final String RANGE_KEY = "sk";
	public static final String GSI_ONE_NAME = "gsiOne";
	public static final String GSI_ONE_HASH_KEY = "gsiOnePk";
	public static final String GSI_ONE_RANGE_KEY = "gsiOneSk";
	public static final String GSI_TWO_NAME = "gsiTwo";
	public static final String GSI_TWO_HASH_KEY = "gsiTwoPk";
	public static final String GSI_TWO_RANGE_KEY = "gsiTwoSk";
	public static final String LSI_ONE_NAME = "lsiOne";
	public static final String LSI_ONE_RANGE_KEY = "lsiOneSk";
	public static final String LSI_TWO_NAME = "lsiTwo";
	public static final String LSI_TWO_RANGE_KEY = "lsiTwoSk";
	public static final String LSI_THREE_NAME = "lsiThree";
	public static final String LSI_THREE_RANGE_KEY = "lsiThreeSk";
}

package com.jessica.dynamodb.utils;

public class KeyGenerator {
	public static final String SEPARATOR = "#";

	private KeyGenerator() {
	}

	public static String createHashKey(String itemName, String... hashKeys) {
		StringBuilder hashKeyBuilder = new StringBuilder(itemName);
		for (String hashKey : hashKeys) {
			hashKeyBuilder.append(SEPARATOR).append(hashKey);
		}
		return hashKeyBuilder.toString();
	}

	public static String createRangeKey(String... rangeKeys) {
		StringBuilder rangeKeyBuilder = new StringBuilder();
		for (int i = 0; i < rangeKeys.length; i++) {
			rangeKeyBuilder.append(rangeKeys[i]);
			if (i < rangeKeys.length - 1) {
				rangeKeyBuilder.append(SEPARATOR);
			}
		}
		return rangeKeyBuilder.toString();
	}

	public static String[] parseHashKey(String itemName, String hashKey) {
		String keySubStr = hashKey.substring(itemName.length());
		return keySubStr.split(SEPARATOR);
	}

	public static String[] parseRangeKey(String rangeKey) {
		return rangeKey.split(SEPARATOR);
	}
}

创建表

根据aws dynamodb的最佳实践: As a general rule, you should maintain as few tables as possible in a DynamoDB application.

所以这里我们只需要创建一个dynamodb表(develop.Favorite), 三个dto结构共用同一张表.

综上分析,该表需要有两个Global Secondary Index和三个 Local Secondary Index,由于三个Dto共用同一张表,并且分区键和排序键都是多个字段组合的形式,所以表的分区键以pk(partion key)表示,排序键以sk(sort key)表示,Secondary Index和名字以及分区键和排序键都采用通用的命名方式,比如gsiOne, gsiOnePk,gsiOneSk.

创建serverless.yml文件,使用sls deploy -v来进行部署.

service: jessica-favorite-dto
                 
provider:
  name: aws
  stage: ${opt:stage, 'develop'}
  region: ${opt:region, 'ap-southeast-1'}
  stackName: ${self:provider.stage}-${self:service}

resources:
  Resources:
    FavoriteTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: ${self:provider.stage}.Favorite
        AttributeDefinitions:
          - AttributeName: pk
            AttributeType: S
          - AttributeName: sk
            AttributeType: S
          - AttributeName: gsiOnePk
            AttributeType: S
          - AttributeName: gsiOneSk
            AttributeType: S
          - AttributeName: gsiTwoPk
            AttributeType: S
          - AttributeName: gsiTwoSk
            AttributeType: S
          - AttributeName: lsiOneSk
            AttributeType: S
          - AttributeName: lsiTwoSk
            AttributeType: S
          - AttributeName: lsiThreeSk
            AttributeType: S
        KeySchema:
          - AttributeName: pk
            KeyType: HASH
          - AttributeName: sk
            KeyType: RANGE
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        BillingMode: PROVISIONED
        LocalSecondaryIndexes:
          - IndexName: lsiOne
            KeySchema:
              - AttributeName: pk
                KeyType: HASH
              - AttributeName: lsiOneSk
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
          - IndexName: lsiTwo
            KeySchema:
              - AttributeName: pk
                KeyType: HASH
              - AttributeName: lsiTwoSk
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
          - IndexName: lsiThree
            KeySchema:
              - AttributeName: pk
                KeyType: HASH
              - AttributeName: lsiThreeSk
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
        GlobalSecondaryIndexes:
          - IndexName: gsiOne
            KeySchema:
              - AttributeName: gsiOnePk
                KeyType: HASH
              - AttributeName: gsiOneSk
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
            ProvisionedThroughput:
              ReadCapacityUnits: 1
              WriteCapacityUnits: 1
          - IndexName: gsiTwo
            KeySchema:
              - AttributeName: gsiTwoPk
                KeyType: HASH
              - AttributeName: gsiTwoSk
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
            ProvisionedThroughput:
              ReadCapacityUnits: 1
              WriteCapacityUnits: 1
        TimeToLiveSpecification: 
          AttributeName: ttl
          Enabled: true
        PointInTimeRecoverySpecification:
          PointInTimeRecoveryEnabled: false
        SSESpecification:
          SSEEnabled: false
        ContributorInsightsSpecification:
          Enabled: false
        Tags:
          - Key: product
            Value: ${self:service}

完整代码

GitHub - JessicaWin/dynamodb-in-action: introduce usage of dynamodb

码农公寓

案例

表设计

TagDto

FavoriteDataDto表

FavoriteDataTagDto

工具类

创建表

完整代码

相关文章