Amazon DynamoDB

AWS Tips I Wish I'd Known Before I Started


  • DynamoDb is a noSQL schema-less storage. Each entity can have different attributes. Total item/entity size 64 KB (UTF-8), name and value count towards size
  • Maximum tables 256/region/account, can request limit increase
  • No practical limit in table size, bytes or items
  • Attributes are name=value pairs and can be single value or multi-value set
  • Attribute types: String, binary, number (38 digits of precision after the decimal point, and can be between 10^-128 to 10^+126), or sets, no empty values or sets
  • Idempotent conditional updates: only applied if conditions are met, safe to retry in the event of failure to acknowledge
  • Atomic counters: increment/decrement, not idempotent, use conditional for idempotent
  • Consistent reads consume more resources, time, and cost more money (avoid them when possible)
  • Must create indexes when tables are created, can't update, add or delete later
  • Hash key distributes data among resources
  • Range key partitions data
  • Query is more efficient than scan, scan reads every record in the table
  • Scanning always scans the entire table, slows as table grows, can consume entire provisioned throughput in a single operation
  • Scan imposes 1 MB limit before filtering
  • Must pre-configure read/write capacity for tables when created
  • Automatic result pagination in 1 MB increments
  • Result limits can be imposed
  • Scans by default are sequential, can specify degree of parallelism
  • Best Practices

Indexes

  • Must create indexes during table create, can't update, add or delete later

Local Secondary Indexes

  • Hash must be same as tables PK hash plus some other table attribute for a range key
  • Each hash key is limited to 10 GB for all indexed items
  • Can only query over single partition based on range key
  • Queries can be either eventual or strong consistency
  • Can request data not projected in the index
Global Secondary Indexes
  • Hash and range can be any table attribute
  • No limit on size
  • Can query over entire table
  • Queries can only be eventual consistency
  • Can only request data projected in the index

Comments

Post a Comment

Popular posts from this blog

Sites, Newsletters, and Blogs

BLOGS Java.net Weblogs Martin Fowler's blog Paul Graham Ruminations of a Programmer Form Follows Function Newsletters Core Java Tech Tips Enterprise Java Tech Tips JavaSpecialists.EU TheServerSide.com Sites Java.net Today Hot Java Sites CSS Zen Garden Evernote Hacker News Pragmatic Programmer
Read more

Oracle JDBC ReadTimeout QueryTimeout

If a query exceeds the oracle.jdbc.ReadTimeout without receiving any data, an exception is thrown and the connection is terminated by the Oracle driver on the client. Unfortunately the session will still be queued on the database and continue to wait for locks, hold any current locks, and complete any DML/PL*SQL procedures that are pending on the server-side of the orphaned connection. If an application, on a another connection, due to ReadTimeout exception, retries DML/PL*SQL  which requires locks, those queries will queue behind the initial DML/PL*SQL. This will increase the workload exacerbating the situation. On applications with retries, this can be observed by querying the v$session table  or gv$session on RAC and noting new sessions started periodically based on the ReadTimeout interval. The sessions may often have the same SQL_ID and/or SQL_HASH_VALUE. If stmt.setQueryTimeout(Seconds) is issued and the statement exceeds the timeout, it will atte...
Read more

Locks held on Oracle for hours after sessions abnormally terminated by node failure

If a session holds db locks and is abnormally terminated (no fin/ack), the locks will persist until the db session is closed, typically around 2 hours and 12 minutes with default network tcp_keepalive settings. Abnormally terminated does not include CTRL C or kill -9. The ojdbc6 driver apparently has a shutdown hook thread that closes the connections in graceful shutdowns. The OS apparently closes the connections when processes are killed. Abnormally terminated might include power failure, firewall failures, Out Of Memory, OS/kernel crash, network connection failure, JBoss or other server node failure, etc. This was easily reproduced by creating a  process that connected to the db and updated a record but did not commit. While the first process was waiting, a second process with a contending update was started which blocked on the first update. The first clients network cable was then disconnected. The second clients transaction waited for approximately 2 hours until the databa...
Read more