Query

Queries are the most efficient way to retrieve items from DynamoDB, which is why it is recommend to be very thoughtful when modeling your indexes.

Example Setup

Table Definition

{
  "TableName": "electro",
  "KeySchema": [
    {
      "AttributeName": "pk",
      "KeyType": "HASH"
    },
    {
      "AttributeName": "sk",
      "KeyType": "RANGE"
    }
  ],
  "AttributeDefinitions": [
    {
      "AttributeName": "pk",
      "AttributeType": "S"
    },
    {
      "AttributeName": "sk",
      "AttributeType": "S"
    },
    {
      "AttributeName": "gsi1pk",
      "AttributeType": "S"
    },
    {
      "AttributeName": "gsi1sk",
      "AttributeType": "S"
    }
  ],
  "GlobalSecondaryIndexes": [
    {
      "IndexName": "gsi1pk-gsi1sk-index",
      "KeySchema": [
        {
          "AttributeName": "gsi1pk",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "gsi1sk",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL"
      }
    },
    {
      "IndexName": "gsi2pk-gsi2sk-index",
      "KeySchema": [
        {
          "AttributeName": "gsi2pk",
          "KeyType": "HASH"
        },
        {
          "AttributeName": "gsi2sk",
          "KeyType": "RANGE"
        }
      ],
      "Projection": {
        "ProjectionType": "ALL"
      }
    }
  ],
  "BillingMode": "PAY_PER_REQUEST"
}

Example Entity

import DynamoDB from "aws-sdk/clients/dynamodb";
import { Entity } from "electrodb";

const client = new DynamoDB.DocumentClient();

const table = "electro";

const StoreLocations = new Entity(
  {
    model: {
      service: "MallStoreDirectory",
      entity: "MallStore",
      version: "1",
    },
    attributes: {
      cityId: {
        type: "string",
        required: true,
      },
      mallId: {
        type: "string",
        required: true,
      },
      storeId: {
        type: "string",
        required: true,
      },
      buildingId: {
        type: "string",
        required: true,
      },
      unitId: {
        type: "string",
        required: true,
      },
      category: {
        type: [
          "spite store",
          "food/coffee",
          "food/meal",
          "clothing",
          "electronics",
          "department",
          "misc",
        ],
        required: true,
      },
      leaseEndDate: {
        type: "string",
        required: true,
      },
      rent: {
        type: "string",
        required: true,
        validate: /^(\d+\.\d{2})$/,
      },
      discount: {
        type: "string",
        required: false,
        default: "0.00",
        validate: /^(\d+\.\d{2})$/,
      },
      tenants: {
        type: "set",
        items: "string",
      },
      warnings: {
        type: "number",
        default: 0,
      },
      deposit: {
        type: "number",
      },
      contact: {
        type: "set",
        items: "string",
      },
      rentalAgreement: {
        type: "list",
        items: {
          type: "map",
          properties: {
            type: {
              type: "string",
            },
            detail: {
              type: "string",
            },
          },
        },
      },
      petFee: {
        type: "number",
      },
      fees: {
        type: "number",
      },
      tags: {
        type: "set",
        items: "string",
      },
    },
    indexes: {
      stores: {
        pk: {
          field: "pk",
          composite: ["cityId", "mallId"],
        },
        sk: {
          field: "sk",
          composite: ["buildingId", "storeId"],
        },
      },
      units: {
        index: "gsi1pk-gsi1sk-index",
        pk: {
          field: "gsi1pk",
          composite: ["mallId"],
        },
        sk: {
          field: "gsi1sk",
          composite: ["buildingId", "unitId"],
        },
      },
      leases: {
        index: "gsi2pk-gsi2sk-index",
        pk: {
          field: "gsi2pk",
          composite: ["storeId"],
        },
        sk: {
          field: "gsi2sk",
          composite: ["leaseEndDate"],
        },
      },
    },
  },
  { table, client },
);

(Example code on this page that references the entity StoreLocations uses the following Entity and Table Definition found below)

Query Items

ElectroDB queries use DynamoDB’s query method to retrieve items based on your table’s indexes.

Forming a composite Partition Key and Sort Key is a critical step in planning Access Patterns in DynamoDB. When planning composite keys, it is crucial to consider the order in which they are composed. As of the time of writing this documentation, DynamoDB has the following constraints that should be taken into account when planning your Access Patterns:

1. You must always supply the Partition Key in full for all queries to DynamoDB.
2. You currently have the following operators available to narrow results via your Sort Key
3. To act on single record, you will need to know the full Partition Key and Sort Key for that record.

Providing composite attributes to queries

While ElectroDB does abstract out the formatting of your index keys, you will still need to provide ElectroDB with the values necessary to build your keys. The following examples use the StoreLocations Entity defined in the Example Setup section above.

Good: Includes at least the PK

await StoreLocations.query
  .stores({
    cityId: "Atlanta1",
    mallId: "EastPointe",
  })
  .go();

Good: Includes at least the PK, and the first SK attribute

await StoreLocations.query
  .stores({
    cityId: "Atlanta1",
    mallId: "EastPointe",
    buildingId: "f34",
  })
  .go();

Good: Includes at least the PK, and the all SK attributes

await StoreLocations.query
  .stores({
    cityId: "Atlanta1",
    mallId: "EastPointe",
    storeId: "LatteLarrys",
    buildingId: "f34",
  })
  .go();

Bad: No PK composite attributes specified

With DynamoDB, queries require at least a Partition Key is provided. Without specifying the partition key composite attributes (cityId and mallId), ElectroDB is unable to build a key to supply to DynamoDB. This example will throw.

await StoreLocations.query.stores().go();

Bad: Not All PK Composite Attributes included

The ElectroDB cannot form a Partition Key without a cityId, this example will throw.

await StoreLocations.query
  .stores({
    mallId: "EastPointe",
  })
  .go();

Bad: Composite Attributes not included in order

This example will _not throw, but will it ignore unitId because storeId was not supplied and because ElectroDB cannot logically build a valid sort key string without the all values being provided left to right.

await StoreLocations.query
  .stores({
    cityId: "Atlanta1",
    mallId: "EastPointe",
    unitId: "B24",
  })
  .go();

Using composite attributes to make hierarchical keys

Carefully considering the sequential order Composite Attributes are defined will allow ElectroDB to express hierarchical relationships and unlock more available Access Patterns for your application.

For a comprehensive and interactive guide to build queries please visit this runkit: https://runkit.com/tywalch/electrodb-building-queries.

Access Pattern Queries

When you define your indexes in your model, you are defining the Access Patterns of your entity. The composite attributes you choose, and their order, ultimately define the finite set of index queries that can be made. The more you can leverage these index queries the better from both a cost and performance perspective.

Unlike Partition Keys, Sort Keys can be partially provided. We can leverage this to multiply our available access patterns and use the Sort Key Operations: begins, between, lt, lte, gt, and gte. These queries are more performant and cost-effective than filters. The costs associated with DynamoDB directly correlate to how effectively you leverage Sort Key Operations.

For a comprehensive and interactive guide to build queries please visit this runkit: https://runkit.com/tywalch/electrodb-building-queries.

Partition Key Composite Attributes

All queries require (at minimum) the Composite Attributes included in its defined Partition Key. Composite Attributes you define on the Sort Key can be partially supplied, but must be supplied in the order they are defined.

Composite Attributes must be supplied in the order they are composed when invoking the Access Pattern*. This is because composite attributes are used to form a concatenated key string, and if attributes supplied out of order, it is not possible to fill the gaps in that concatenation.

Sort Key Operations

Begins With Queries

One important consideration when using Sort Key Operations to make is when to use and not to use “begins”.

It is possible to supply partially supply Sort Key composite attributes. Sort Key attributes must be provided in the order they are defined, but it’s possible to provide only a subset of the Sort Key Composite Attributes to ElectroDB. By default, when you supply a partial Sort Key in the Access Pattern method, ElectroDB will create a beginsWith query. The difference between that and using .begins() is that, with a .begins() query, ElectroDB will not post-pend the next composite attribute’s label onto the query.

The difference is nuanced and makes better sense with an example, but the rule of thumb is that data passed to the Access Pattern method should represent values you know strictly equal the value you want.

The following examples will use the following Access Pattern definition for units:

{
  "units": {
    "index": "gsi1pk-gsi1sk-index",
    "pk": {
      "field": "gsi1pk",
      "composite": ["mallId"]
    },
    "sk": {
      "field": "gsi1sk",
      "composite": ["buildingId", "unitId"]
    }
  }
}

The names you have given to your indexes on your entity model/schema express themselves as “Access Pattern” methods on your Entity’s query object:

// Example #1, access pattern `units`
StoreLocations.query.units({ mallId, buildingId }).go();
// -----------------------^^^^^^^^^^^^^^^^^^^^^^

Data passed to the Access Pattern method is considered to be full, known, data. In the above example, we are saying we know the mallId, buildingId and unitId.

Alternatively, if you only know the start of a piece of data, use .begins():

// Example #2
StoreLocations.query.units({ mallId }).begins({ buildingId }).go();
// ---------------------------------^^^^^^^^^^^^^^^^^^^^^

Data passed to the .begins() method is considered to be partial data. In the second example, we are saying we know the mallId and buildingId, but only know the beginning of unitId.

For the above queries we see two different sort keys:

1. "$mallstore_1#buildingid_f34#unitid_"
2. "$mallstore_1#buildingid_f34"

The first example shows how ElectroDB post-pends the label of the next composite attribute (unitId) on the Sort Key to ensure that buildings such as "f340" are not included in the query. This is useful to prevent common issues with overloaded sort keys like accidental over-querying.

The second example allows you to make queries that do include buildings such as "f340" or "f3409" or "f340356346".

For these reasons it is important to consider that attributes passed to the Access Pattern method are considered to be full, known, data.

Examples

The following examples use the Entity and table configuration defined in the section Example Setup above.

Lease Agreements by StoreId

await StoreLocations.query.leases({ storeId: "LatteLarrys" }).go();

Lease Agreement by StoreId for March 22nd 2020

await StoreLocations.query
  .leases({
    storeId: "LatteLarrys",
    leaseEndDate: "2020-03-22",
  })
  .go();

Lease agreements by StoreId for 2020

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .begins({ leaseEndDate: "2020" })
  .go();

Lease Agreements by StoreId after March 2020

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .gt({ leaseEndDate: "2020-03" })
  .go();

Lease Agreements by StoreId after, and including, March 2020

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .gte({ leaseEndDate: "2020-03" })
  .go();

Lease Agreements by StoreId before 2021

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .lt({ leaseEndDate: "2021-01" })
  .go();

Lease Agreements by StoreId before February 2021

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .lte({ leaseEndDate: "2021-02" })
  .go();

Lease Agreements by StoreId between 2010 and 2020

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .between({ leaseEndDate: "2010" }, { leaseEndDate: "2020" })
  .go();

Lease Agreements by StoreId after, and including, 2010 in the city of Atlanta and category containing food

await StoreLocations.query
  .leases({ storeId: "LatteLarrys" })
  .gte({ leaseEndDate: "2010" })
  .where(
    (attr, op) => `
      ${op.eq(attr.cityId, "Atlanta1")} AND ${op.contains(
        attr.category,
        "food",
      )}
  `,
  )
  .go();

Execution Options

Query options can be added the .params() and .go() to change query behavior or add customer parameters to a query.

By default, ElectroDB enables you to work with records as the names and properties defined in the model. Additionally, it removes the need to deal directly with the docClient parameters which can be complex for a team without as much experience with DynamoDB. The Query Options object can be passed to both the .params() and .go() methods when building you query. Below are the options available:

{
  params?: object;
  table?: string;
  data?: 'raw' | 'includeKeys' | 'attributes';
  pager?: 'raw' | 'cursor';
  originalErr?: boolean;
  response?: "default" | "none" | "all_old" | "updated_old" | "all_new" | "updated_new";
  ignoreOwnership?: boolean;
  limit?: number;
  pages?: number | 'all';
  logger?: (event) => void;
  listeners Array<(event) => void>;
  attributes?: string[];
  order?: 'asc' | 'desc';
};