Wow: Multiple Identity Providers and AWS Cognito

I've finally found time to experiment with multiple identity providers for Cognito. Mostly to understand how a CognitoId is formed, merged, invalidated. It turns out this is a significant finding, especially when this Id is used, say, as a primary key for data storage!

Recall, the original sensor and sensor2 projects were plumbed with Login With Amazon as the identity provider to Cognito. This new experiment adds GooglePlus as a second provider. Here you can see the test platform on the iPhone:

Keep in mind that for this sensor2 application, the returned CognitoId is used as the customer's key into the storage databases. Both for access control and as the DynamoDB hash key.

The flow on the iPhone goes roughly as follows:

• A user can login via one or both of the providers
• A user can logout
• A user can also login using same credentials on a different devices (e.g. another iPhone with the application loaded)

Now here's the interesting part. Depending on the login ordering, the CognitoId returned to the application (on the watch in this case) can change! Here's how it goes with my test application (which includes "Logins" merge)

• Starting from scratch on a device
• Login via Amazon where user's Amazon identity isn't known to this Cognito pool:
• User will get a new CognitoId allocated
• If user logs out and logs back in via Amazon, the same Id will be returned
• If the user now logs into a second device via Amazon, the same Id will be returned
• (so far this makes complete sense)
• Now, if the user logs out and logs in via Google, a new Id will be returned
• Again, if the user logs out and in again and same on second device, the new Id will continue to be returned
• (this all makes sense)
• At this point, the system thinks these are two users and those two CognitoIds will be used as different primary keys into the sensor database...
• Now, if the user logs in via Amazon and also logs in via Google, a CognitoId merge will occur
• One, or the other of those existing Ids from above will be returned
• And, the other Id will be marked via Cognito as disabled
• This is a merge of the identities
• And this new merge will be returned on other devices from now on, regardless of whether they log in solely via Amazon or Google
• (TODO: what happens if user is logged into Amazon, has a merged CognitoId and then they log in using a second Google credential?)

This is all interesting and sort of makes sense -- if a Cognito context has a map of logins that have been associated, then Cognito will do the right thing. This means that some key factors have to be considered when building an app like this:

• As with my application, if the sensor database is keyed by the CognitoId, then there will be issues of accessing the data indexed by the disabled CognitoId after a merge
• TODO: will this happen with multiple devices going through an anonymous -> identified flow?
• It may be that additional resolution is needed to help with the merge -- e.g. if there is a merge, then ask the user to force a join -- and then externally keep track of the merged Ids as a set of Ids -> primary keys for this user...

Anyway, I'm adding in a couple more providers to make this more of a ridiculous effort. After which I'll think about resolution strategies.

TestFlight version of sensor2 is available in Apple Store

The current version of sensor2 is available as a test flight application if you'd like to try it out. This will give you a quick try of the basic sensor capture and display in the minimal website.

Recall it uses Amazon Login as the sole identity provider right now.

Drop me a note, or comment here if you'd like to try the TestFlight version.

sensor2 code cleanup -- you can try it too

After a bit of field testing, I've re-organized the sensor2 code to be more robust. Release tag for this change is here. Major changes include:

• The Watch still sends CMSensorRecorder data directly to DynamoDB
• However, the Watch now asks the iPhone for refreshed AWS credentials (since the AWS SDK isn't yet working on Watch, this avoids having to re-implement Cognito and login-with-amazon). This means that with today's code, the Watch can be untethered from the iPhone for up to an hour and can still dequeue records to DynamoDB (assuming the Watch has Wi-Fi access itself)
• If the Watch's credentials are bad, empty or expired and Watch can't access the iPhone or the user is logged out of the iPhone part of the app, then Watch's dequeuer loop is stopped
• Dependent libraries (LoginWithAmazon) are now embedded in the code
• A 'logout' on the phone will invalidate the current credentials on the Watch

This code should now be a bit easier to use for reproducing my experiments. Less moving parts, simpler design. I'll work on the README.md a bit more to help list the steps to set up.

And finally, this demonstrates multi-tenant isolation of the data in DynamoDB. Here's the IAM policy for logged in users:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "mobileanalytics:PutEvents",
                "cognito-sync:*",
                "cognito-identity:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Sid": "Stmt1449552297000",
            "Effect": "Allow",
            "Action": [
                "dynamodb:BatchWriteItem",
                "dynamodb:UpdateItem",
                "dynamodb:Query"
            ],
            "Resource": [
                "arn:aws:dynamodb:us-east-1:499918285206:table/sensor2"
            ],
            "Condition": {
                "ForAllValues:StringEquals": {
                    "dynamodb:LeadingKeys": [
                        "${cognito-identity.amazonaws.com:sub}"
                    ]
                }
            }
        }
    ]
}

In the above example the important lines are the condition -- this condition entry enforces that only rows with HashKey the same as the logged in user's cognitoId will be returned. This is why we can build applications with direct access to a data storage engine like DynamoDB!

You can read the details of IAM+DynamoDB here.

Anyway, back to performance improvements of the dequeue process. Everything is running pretty good, but the Watch still takes a long time to get its data moved.

A note on sensor2 dequeue performance

I've examined sensor2 dequeue performance. Some interesting observations indeed!

• A single dequeue loop (1250 samples for 25 seconds) time takes a bit over 7 seconds
• A little under 1 second of this time is getting data from CMSensorRecorder
• Around 4 seconds is required to prepare this data
• The time to send the samples to DynamoDB depends on the network configuration:
• 3 - 6 seconds when the Watch is proxying the network through iPhone using LTE network (with a few bars of signal strength)
• 2 - 4 seconds when the Watch is proxying the network through iPhone (6s plus) and my home WiFi
• Around 1.5 seconds when the Watch is directly connecting to network using home WiFi

Speeding up the data preparation will help some.  I will set a goal of 1 second:

• Hard coded JSON serializer
• Improvements to the payload signer
• Reduce the HashMap operations (some clever pivoting of the data)

New "serverless" site to explore sensor data

I have updated the UI to parse and render the data from the new data model. You can try it out here.

Recall the data flow is:

• CMSensorRecorder is activated directly on the Watch
• When the application's dequeue is enabled, the dequeued events are:
• Parsed directly into our DynamoDB record format
• Directly sent to DynamoDB from the Watch

And this pure static website directly fetches those records and pivots the data in a vis.js and d3.js format for display.

Next up:

• Get AWS Cognito into the loop to get rid of the long lived AWS credentials
• Work on the iOS framework memory leaks
• Speed up the dequeue (or, resort to a Lambda raw data processor)

Progress: CMSensorRecorder directly to DynamoDB

Relative to before, the pendulum has swung back to the other extreme: a native WatchOS application directly writing to AWS DynamoDB.  Here, we see a screen grab with some events being sent:

This has been an interesting exercise. Specifically:

• With iOS 9.2 and WatchOS 2.1, development has improved
• However, I can't yet get the AWS iOS SDK to work on the Watch directly
• So, I have instead written code that writes directly to DynamoDB
• Including signing the requests
• Including implementing low level API for batchWriteItem and updateItem
• I have also redone the application data model to have a single DynamoDB row represent a single second's worth of data with up to 50 samples per row
• Initially, samples are indexed using named columns (named by the fraction of a second the sample is in)
• Later this should be done as a more general documentDB record
• This approach is a more efficient use of DynamoDB -- provisioning required is around 2 writes/second per Watch that is actively dequeuing (compared to 50 writes/second when a single sample is stored in a row)
• This application also uses NSURLSession directly
• This means that the Watch can send events to DynamoDB using configured WiFi when the iPhone is out of range!
• I have also redone the command loop using GCD dispatch queues (instead of threads)

Anyway, it appears to be doing the right thing. Data is being recorded in CMSensorRecorder, the dequeue loop is processing data and transmitting up to 1250 samples (25 seconds) of data per network call. The custom request generator and call signing are doing the right thing. Perhaps a step in the right direction? Not quite sure:

• I see that the actual on-Watch dequeue processing takes about 6 seconds for 25 seconds worth of data. Since all of the data preparation must occur on the Watch (there is no middle man), the additional work of pivoting the data, preparing the DynamoDB request are borne by the Watch.
• Profiling shows the bulk of this processing time is in JSON serialization!
• Another approach would be minimal processing on the Watch.  e.g. "dump the raw data to S3" and let an AWS Lambda take care of the detailed processing. This is probably the best approach although not the cheapest for an application with many users.
• I'm now running tests long enough to see various memory leaks! I've been spending a bit of time with the memory allocator tools lately...
• I have run into a few with the NSURLSession object
• The JSON serializer also appears to leak memory
• Possibly NSDateFormatter also is leaking memory

Here's what a dequeue loop looks like in the logs. You can see the blocks of data written and the loop processing time:

Jan  3 21:05:11 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: dequeueLoop(1)

Jan  3 21:05:11 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: flush itemCount=23, minDate=2016-01-04T05:01:54.557Z, maxDate=2016-01-04T05:01:54.998Z, length=2621

Jan  3 21:05:12 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: data(Optional("{}"))

Jan  3 21:05:13 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: commit latestDate=2016-01-04 05:01:54 +0000, itemCount=23

Jan  3 21:05:13 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: dequeueLoop(2)

Jan  3 21:05:13 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: flush itemCount=49, minDate=2016-01-04T05:01:55.018Z, maxDate=2016-01-04T05:01:55.980Z, length=5343

Jan  3 21:05:14 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: data(Optional("{}"))

Jan  3 21:05:14 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: commit latestDate=2016-01-04 05:01:55 +0000, itemCount=72

Jan  3 21:05:15 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: dequeueLoop(3)

Jan  3 21:05:20 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: flush itemCount=1250, minDate=2016-01-04T05:01:56.000Z, maxDate=2016-01-04T05:02:20.988Z, length=88481

Jan  3 21:05:23 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: data(Optional("{\"UnprocessedItems\":{}}"))

Jan  3 21:05:23 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: commit latestDate=2016-01-04 05:02:20 +0000, itemCount=1322

Jan  3 21:05:23 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: dequeueLoop(4)

Jan  3 21:05:30 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: flush itemCount=1249, minDate=2016-01-04T05:02:21.008Z, maxDate=2016-01-04T05:02:45.995Z, length=88225

Jan  3 21:05:32 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: data(Optional("{\"UnprocessedItems\":{}}"))

Jan  3 21:05:32 Gregs-AppleWatch sensor2 WatchKit Extension[152] <Warning>: commit latestDate=2016-01-04 05:02:45 +0000, itemCount=2571

And here is what a record looks like in DynamoDB. This shows the columnar encoding of a few of the X accelerometer samples:

I have a checkpoint of the code here. Note that this code is somewhat hard coded for writing only to my one table with only AWS authorizations to write.

TODO:

• Update the UI to help explore this data
• See if there is a more efficient use of the JSON serializer
• Examine some of the framework memory leaks
• Try to speed up the dequeue to be better than 6 seconds of wall clock for 25 seconds of data.

CMSensorRecorder from Watch to AWS

I've done a couple of experiments with the direct networking from the WatchOS 2.0 using NSURLSession. Based on the results, I now have a strategy for getting the iPhone out of the loop.

Recall the current plumbing looks like this:

• CMSensorRecorder data collected in Watch
• Dequeued by a native application on the Watch
• Sent via WCSession to iPhone
• iPhone uses AWS KinesisRecorder to buffer received events
• KinesisRecorder sends to AWS (using Cognito credentials)
• Kinesis is dequeued to an AWS Lambda
• The Lambda stores the records to DynamoDB (using a fixed IAM role)

This is fine, but has a lot of moving parts. And more importantly, Kinesis is provisioned at expected throughput and you pay by the hour for this capacity.

My next experiment will look like this:

• CMSensorRecorder data collected in Watch
• Dequeued by a native application on the Watch
• Sent directly to AWS IoT (using on-watch local long lived IAM credentials)
• An IoT rule will send these events to Lambda
• The Lambda stores the records to DynamoDB (using a fixed IAM role)

The important difference here is that IoT is charged based on actual use, not provisioned capacity. This means the effective costs are more directly related to use and not expected use!

Also, the Watch will be able to directly transmit to AWS instead of going through the iPhone (well, it does proxy through its paired phone if in range, otherwise it goes direct via WIFI if a known network is near). This feature will be an interesting proof of concept indeed.

Anyway, for this first experiment the writes to IoT are via http POST (not via MQTT protocol). And authentication is via long lived credentials loaded into the Watch, not Cognito. This is primarily because I haven't yet figured out how to get the AWS IoS SDK to work on the Watch (and the iPhone at the same time). So, I am instead using low-level AWS Api calls. I expect this will change with a future release of the SDK.

I have toyed with the idea of having the Watch write directly to DynamoDB or Lambda instead of going through IoT. Since the Watch is already buffering the sensor data, I don't really need yet another reliable queue in place. Tradeoffs:

1. Send via IoT
• + Get Kinesis-like queue for reliable transfer
• + Get some runtime on IoT and its other capabilities
• - Paying for another buffer in the middle
2. Send direct to Lambda
• + One less moving part
• - To ensure sending data, need to make a Synchronous call to Lambda which can be delayed when writing to DynamoDB, not sure how well this will work on networks in the field
3. Send direct to DynamoDB
• + The lowest cost and least moving parts (and lowest latency)
• - DynamoDB batch writes can only handle 25 items (0.5 seconds) of data

Note: on DynamoDB, earlier I had discussed a slightly denormalized data storage scheme. One where each second of data is recorded on one dynamoDB row (with separately named columns per sub-second event). Since DynamoDB can do no-clobber updates, this is a nice tradeoff of rows vs data width. This would change the data model and the reader would need to take this into account, but this may make the most sense no matter what. Basically doing this gets better utilization of a DynamoDB 'row' by compacting the data as much as possible. This probably reduces the overall cost of using DynamoDB too as the provisioning would, in general, be reduced. So, I may just re-do the data model and go direct to DynamoDB for this next POC.

Stay tuned!