AWS Lake Formation

Michael | 20 August 2019

AWS continues to raise the bar across a whole lot of technology segments and in AWS Lake Formation they have created a one-stop shop for the creation of Data Lakes. As always, AWS is further abstracting their services to provide more and more customer value. The evolution of this process can be seen by looking at AWS Glue.

read more

Creating SSM Parameters

Michael | 27 June 2019

There are many services that I really like in AWS but one of those that doesn’t get a lot of attention is AWS Systems Manager. While I can’t say that I use all the features regularly of this service, two that I use a lot are the Session Manager and also the parameter store.

read more

S3 Events and Error 403

Michael | 26 June 2019

I have been performing some data transformations using Lambda and S3 Events and for certain S3 keys I noticed that I was getting an Error 403 in my code:

read more

Backup for SQL Server RDS

Michael | 15 December 2018

Earlier I described how I performed a restore of a database into Amazon RDS for SQL Server. While I don’t play in SQL Server a lot I had a different requirement from another client a few weeks later that required me to perform a backup of an existing RDS instance.

read more

Restoring SQL Server Databases in RDS from S3

Michael | 19 November 2018

I often don’t play in RDS that much but every once in a while I jump back into it when a clients needs me to do so. I know that there is a way of restoring a SQL Server database from an S3 location, but for the life of me I could remember how it was done. For added bonus points I decided that I should try interacting with the database completely without a windows host.

read more

PySpark and Glue together

Michael | 28 August 2018

I have been playing around with Spark (in EMR) and the Glue Data Catalog a bit and I really like using them together. The ability of you being able to use EMR to transform the data and then being able to query it in either Spark, Glue or Athena - and through Athena via a JDBC data source is a real winner.

read more