Unsupported case of DataType: com.amazonaws.services.glue.schema.types.StringType@e7b95c9 and DynamicNode: longnode

Question

I am trying to extract 27 DynamoDB tables from a single Database using the Visual editor in AWS Glue. I have successfully crawled the database and my workflow for the job is.

1. Extract from Source table (DynamoDB).
2. Apply Transform (usually 1:1 and leaving all datatypes as default)
3. Store in S3 Bucket
4. Create a new table in Catalog (orginal table names have ugly GUID etc so this is to just neaten up).

I have so far managed to create 6 parallel tasks in the visual editor which work successfully and i can query the data in Athena, however a couple of my tables are throwing errors which appear to be dataype mismatch such as "An error occurred while calling o385.pyWriteDynamicFrame. Unsupported case of DataType: com.amazonaws.services.glue.schema.types.StringType@e7b95c9 and DynamicNode: longnode."

I can't seem to get any sense from the stack trace on which part of the script is falling down or more specifically which field is giving me trouble. I've crudely tried to jjust drop any fields with "long" datatype to unblock the error but to avail.

Is anyone able to give me pointers how to debug which field is causing the problem?

Answer 1

This means that were a string is expected long datatype is being found. Inspect your columns with string datatype and convert any long values in such to string.