This blog post is the first of my new blog post series that will be published in the following weeks. The aim is to cover getting certificated on all the major cloud platforms currently (1/2019) in Europe: Amazon Webservices (AWS), Google Cloud Platform (GCP) and Microsoft Azure. I’ve done my Pro level certs on all these platforms quite recently (within a year) so I’ve some knowledge on the issue. I’ve written texts similar to this on our internal wiki, but as there are no secrets in there, I decided to re-write the material in more reader friendly format (and less like a stream of consciousness).
Like the good authors of certification guides I’m not claiming that you will get certified by doing what I advice you to, but I can safely say that following my advice raises your probability of success.
This first blog post is labelled ‘introduction’. I will cover general stuff about certifications, suggested path for going through the clouds (for the hardcore “gotta get ’em all”-cloud people out there) and also some general notes on preparing for the certifications. The later posts will each focus on one of the cloud platforms.
The three musketeers
Correlation between getting a certification and knowing your stuff
Before going to the actual how-to part, let’s think for a while what a certificate actually is and does getting one hold any significance. Being a holder of a certificate means that you have passed an exam where your general knowledge on the platform has been tested. Depending on your path (development, architecture etc.) your knowledge goes a little bit deeper on certain areas, but you most likely need to know the same basic stuff on all associate-level certs for a single platform. For pro level exams, it means that you also possess some deeper level of knowledge on the subject and also possess problem-solving skills giving you the title Pro; a professional proficiency on the subject. This is not to be mistaken with a guru. Does a certificate make you a cloud engineer, to be quickly hired and put to a challenging customer project? Pro level cert certainly would imply that, but associate? No. An associate cert is a first stepping stone, meaning that you know some rules and best practices on the subject, but without any elbow grease on the platform, it amounts only to a good start. Of course, you can just put your study-cap on and study like possessed and pass a pro exam without never even launching a single instance, but I dare to say that it’s quite an uncommon scenario.
Why bother with associate certifications then? Well, as said previously, it indicates that you know the best practices of the platform, and while that might not land you your dream job, it’s still a quite big deal. When working in a cloud environment it’s very easy to deploy applications and create virtual machines, but it’s also really easy to do them wrong, using architecture not fit for cloud-age or in a worst case compromising security. Yes, you could just watch the videos and read the documents and be equally knowledgeable on the subject as someone with a certificate, but if you took all that time to study, why not do the certification when you are on it.
Clouds and order of conquest
If your work or side projects do not involve using any of the platforms and you are totally free to choose where to begin, I would (once again) pick AWS. AWS is by far the market leader and mastering it still opens more doors than GCP and Azure combined. If you are more curious for example about GCP, pick that. In studying practicality falls second to motivation.
If you really have a lot of free time on your hands and want to get certified on all the cloud platforms you can start wherever you want… but if you start with either AWS or GCP, do the remaining one before going for Azure. Terminology- and function-wise AWS and GCP are quite similar to the extent that Google has published even a quite handy cross-reference document from AWS experts to grasp their platform. Where terms for higher levels of abstraction are also similar for Azure, such as block storage and object storage the Microsoft way of doing cloud is still quite different. Understanding Azure requires you to forget how stuff is done in other cloud platforms and learn the Azure way. I did the AWS → Azure → GCP trip and cannot recommend it to anyone.
Why use one sentence to describe the cloud platform study order, when you can confuse the ***t out of people with a diagram
An easyish path for studying for a certification goes like this
- Read the certification requirements
- Do some web course that goes through the relevant material
- Read the documentation for most important services
- Do some practice exams
- Ace the exam
Certification requirements and service documentation will be produced by the cloud platform organization and readable on their websites. Web courses and practice exams are usually provided by some third party, I will give hints on good places for platform-specific blogs of this series.
If you spend one hour daily, you should be able to do your first cloud certification in two months, even without previous experience. I suggest that before trying any of the Pro level certifications you get hands-on experience with some cloud platform for at least one year. It does not have to be that specific platform, as usually on those exams emphasis is more on “cloud thinking” and less on trivia.
Even if there are differences in how the exams are done on different platforms, there are some universal strategies:
- Book your exam when you start studying
- It works as a goal for your studies, giving that small ‘oomph’ to your motivation
- In the exam, don’t get stuck, time is of the essence
- Mark the hard ones and come back later
- Don’t overstress
- Even if you fail, you can always try again. You’ll also benefit from failure: now you know your weak points and can improve on that
Follow-up posts in this series:
Today was the last day of the conference and it’s starting to show. People are heading home, so sessions are not that crowded and last session ends around noon / 1pm. People wearing conference badges are thinning out and replaced with more regular tourists.
I managed to get into a very good chalk talk about Cloudformation given by Check Meyer and Ryan Lohan. So a big thanks to them! We had a good discussion about tooling, feature requests and so on. This is also something that many people might overlook. AWS prioritizes features and their implementation on the basis of feedback received from customers. You do not have to be APN partner, done certifications, or anything like that. As Amazon/AWS themselves say, they try to be the most customer-centric company there is. The most important thing is that instead of silently contemplating on a feature or bug you should make yourself heard. Join AWS Developers slack. Use Twitter, AWS forums, email their evangelists or talk to their employees at any event. If the barrier is still too big you can email me or my colleagues and we can bring your case to AWS. Make yourself heard!
Finally, some tips & tricks in case you find yourself in Re:Invent 2019!
First, don’t be too greedy. There are tons of good sessions but the thing is – the sessions are recorded and can be watched at a later time Youtube. Chalk talks, workshops and hackathons are not. You get to talk to product-teams or their representatives in those. I can highly recommend attending those and if there are competing sessions at the same time try to prioritize chalk-talks and workshops higher than breakout sessions.
Second, as I’ve mentioned in the first post, Las Vegas is designed to remove your money from you. There will be coffee/soda/water provided by Re:Invent as well as some snacks. The expo area is excellent if you want to eat something. There is usually food being served. Hotels are expensive and if you need to buy something there are multiple Walgreens (shops) on the Strip.
Third, keynotes by Andy Jassy and Werner Vogels are great. However, you should consider passing the keynotes if there are some other interesting happening at the same time. For example hackathons or gamedays. Keynotes are usually recorded and any announcements made are also published on Twitter, blogs and so on.
Fourth, when booking your schedule try to cluster up the sessions/workshops/etc you are attending. Moving from one venue to another takes time. Cluster your sessions on certain venues. For example, The Mirage and Venetian are very close to each other. Moving between them is much easier than moving from Mira/Venetian to Aria/Vdara. On the other hand, Aria/Vdara/MGM are situated relatively close to each other.
Fifth, pick your parties. There are TONS of different parties hosted by AWS partners. You cannot visit all so choose early.
Sixth, talk to people. That might not be the easiest thing to do especially for us Finnish whose culture is not extroverted. “Hey, my name is Aki. What do you with AWS?” and the conversation takes on.
Now it’s time to sign-off and get some rest before starting the long trip back home. Quoting Werner now it’s time to “Go build”.
Jeff Bar, Abby Fuller and Simon Elisha before Twitch live
Today was Werner’s turn and boy he didn’t disappoint. The keynote was packed with some very welcome announcements. Again some of the announcements might be missing from the post but those can be found on Twitter, AWS blogs and from the news.
As usual, Werner used a good portion of the keynote to emphasize how critical it is to have control over ones’ infrastructure. To avoid “black boxes” and to prepare for the fact that “everything fails all the time”. By now this shouldn’t be a surprise for anyone and if your architecture is not taking this into account you might be in for some nasty surprises in the future. In order to help companies assume best practices, AWS has a so-called “Well Architected Framework”. This set of guidelines and best practices should be familiar to anyone who is using AWS. Those of you who have done the AWS certifications it is the foundation to learn. Now AWS has come up with “Well Architected” self-service testing tool. It can be used to assess how well your development, operational and governance practices are aligned with the “Well Architected Framework”.
However the announcements today were mostly about serverless computing, namely AWS Lambda. There were some huge updates announced like custom runtimes, layers, ALB support, service integrations with step functions and IDE toolkits. The abstraction level keeps on raising and serverless-computing is becoming more and more mainstream.
To easily sum up all the announcements it is now possible to have your lambda-functions be called by ALB while lambdas themselves can be running Rust, C++ or even Cobol and code can be shared between your functions. Your step functions can interact with other services and you can debug your lambda functions. Additionally API gateway now supports websockets. Streaming data has also become mainstream (pun intended) and even though AWS has Kinesis they announced managed Kafka. Running Kafka at scale is no trivial task so this should be a relief for anyone using Kafka but not wanting to handle the maintenance it requires.
Building systems without any servers at all is now much more feasible and serverless should nowadays be given very careful consideration when starting new projects. It could be said that serverless is a valid option for new development activities and instead of prejudice it should be embraced since serverless/Functions as a Service has come to stay.
Today was a big day. Wednesday morning is usually the time for Andy Jassy (CEO of AWS) to give his keynote. This was the case this year also. The keynote was full of different announcements and it will be quite a task to go through all of them. I’ll leave some of them out and also include some announcements that weren’t in the keynote.
A huge chunk of the talk was about ML. Like Google has their TPU-processors to run ML models, AWS today announced Inferentia processors which should be available next year. Google has a head-start of several years so it is interesting to see how AWS’s offering can match Google’s. In addition to processors there were all kinds of enhancements so if ML is your thing you should definitely read the AWS blog posts about the new features. One thing I’m going to “kehdata” (Sorry English speakers, ‘daring’ is a rough translation of the term, but in Gofore it holds much more meaning. Email me and I’ll explain the concept) is AWS DeepRacer. DeepRacer is radio-controlled car with atom-processor, video-camera, ROS OS and so on. Would definitely be fun way for people to practice ML and reinforcement learning.
Traditionally DynamoDB tables must have had both read capacity and write capacity defined and performance was pretty static in a sense (assuming your data is modelled correctly and you know your access patterns). Then came autoscaling which automatically tunes read/write capacity values based on your traffic. And we have the option for on-demand billing. Based on the blog posts and documentation the on-demand option scales very well right from the start without the need to specify read/write capacity. The cost model is interesting and more closely matches for example Lambda’s model where you only pay for what you use. If your DynamoDB usage is spiky then on-demand might be a very good fit, whereas continuous, huge volume of traffic is much more cost-effective to run on traditional mode where you specify the performance limits yourself.
AWS Control Tower
For several years the best practice has been to distribute applications/services/teams into different AWS accounts and furthermore segregate development, testing and production into different accounts. Natural outcome from this is the fact that the number of AWS accounts in organizations has exploded. So far it has been pretty much DIY-solutions when trying to get overall vision of all your accounts. The bigger the organization, the more they feel pain from this.
Today AWS announced Control Tower which aims to alleviate some of these problems. Automating the set-up of a baseline environment, Control Tower uses existing services like Config, Cloudtrail, Lambda, Cloudwatch, etc. Read more about Control Tower from product page: https://aws.amazon.com/controltower/features/
As an AWS partner our company has a huge number of accounts, so for us Control Tower is a very welcome improvement. We are investigating what it exactly brings to table and where you might still need custom solutions. Stay tuned for more blog posts concentrating solely on Control Tower. Currently it is in preview, so signup and a bit of luck is needed to get early taste of it.
Cloudwatch metrics isn’t exactly new. It has existed a long time and is de-facto solution for metrics collection from AWS services. In addition to Cloudwatch it is very common to see InfluxDB or Prometheus on our clients (usually combined with Grafana for visualization of time-series data).
Today AWS announced Amazon Timestream, a managed time-series database. Targeted solely for time-series data this puts Timestream into direct competition against Prometheus, InfluxDB or Riak TS or Timescale. Naturally this is excellent news if you don’t want to manage servers and want to have your time-series database as a service. No more EC2 instances running Prometheus, no more DIY solutions for HA and so on. AWS mantra has long been that let the “undifferentiated heavy lifting” for them and concentrate on your application and business-logic. Timestream follows this idiom perfectly. Timestream is currently in preview so signup and a bit of luck is needed to test it.
Quantum ledger database
Quantum ledger database and managed blockchain. Well now we have all the buzzwords in one blog. AI/ML handled already and now it is time for blockchain. AWS announced to day two services loosely related to each other, both are currently in preview. Quantum ledger database is database with central trusted authority, immutable append only semantics with the complete history of all the changes ever made. What does it have to do with blockchain? Well, all the changes are chained and cryptographically verified. There is huge amount of use cases! In addition to quantum ledger database AWS also announced managed blockchain which supports Hyperledger Fabric and Ethereum (Hyperledger is first, Ethereum coming later).
There were other new features launched that might stay under the radar if the focus is only on the keynote. One that is very relevant for my current project is the CodeDeploy’s ability to do native blue/green deployments into ECS and Fargate. (more here: https://aws.amazon.com/blogs/devops/use-aws-codedeploy-to-implement-blue-green-deployments-for-aws-fargate-and-amazon-ecs/)
This will definitely be tested out next week.
AWS App Mesh
Also one more nice announcement was AWS App Mesh. Envoy-proxy based service-mesh for EKS, ECS and K8s running on EC2. Like other service meshes the idea is that applications or micro-services do not need to have in-built functionality for service discovery (and possible load-balancing or circuit breaking). Service mesh takes care of it and applications are simpler to implement. App Mesh is in preview but more information can be found on Github: https://github.com/awslabs/aws-app-mesh-examples
Like I said this is not definite list of all the new changes. There are literally tons of new things! Let’s see if Andy left any announcements for Werner tomorrow (hopefully so).
Things are moving fast. Day 2 included a Partner keynote and didn’t contain so much technical announcements. The news in the keynote was mostly about the AWS Marketplace.
AWS introduced “Private marketplace”. Private marketplace allows customers to create a customized catalogue of pre-approved products from the AWS Marketplace. This allows administrators to select only products that are authorized or otherwise meet the criteria decided by your organization. The Private Marketplace can be customized with custom branding. Logo, texts and colour scheme can be changed to match your organization. All controls set up by administrators for the Private Marketplace are applied across AWS Organizations.
This kind of customization and pre-approved catalogues of SKUs can be useful for bigger organizations who wish to have control over what gets deployed. However, using this kind of feature will require vigilance on your offerings through the Private Marketplace. Introducing too much command & control may have a detrimental effect on agility and speed the cloud provides.
In addition to the Private Marketplace, AWS introduced container products in Marketplace. These container-products can be run on ECS, EKS and Fargate and they come in either as task definitions, Helm charts or Cloudformation templates. This announcement brings both VM’s and containers as first-class citizens on the Marketplace and it also offers sellers new options on how to distribute their software.
The marketplace wasn’t the only new fascinating release. Ground Station is service which will communicate with satellites in orbit. This basically means that launching a satellite and talking to it can be accomplished with a very small amount of money compared to the past when in addition to launch costs you would have to build your own ground station (radios, antennas, etc). Universities, schools and companies can now launch satellites if they want to. Space technology is being brought to the public and this will hopefully help to create new innovations and products/services.
I have to admit that “Satellite Communications as a Service” (should it be SCaaS) wasn’t even on the list when I’ve wondered what AWS might publish during the week. There are some caveats in the service though! You will need a Federal Communications Commission (FCC) license and Norad ID of your satellite and you will need to contact AWS in order to activate the service so you cannot just arbitrary book antenna-time and start shooting radio messages to the sky.
Amazon CloudWatch Logs Insights. These announcements bring Kibana-like features to Cloudwatch. It can read multiple formats and especially useful feature is that it autodetects field-names if your logs are JSON-formatted. This feature might reduce the need for ELK-stack. This brings a whole new level on Cloudwatch dashboards.
Finally, it is time to talk about DynamoDB. Today it was announced that DynamoDB now has transactions. Having transaction support makes it possible to use DynamoDB in huge amounts of new use-cases. Now DynamoDB is controversial subject especially among the developers (this is my experience, YMMV). Modelling your data into NoSQL-database is not always straightforward. Developers don’t usually have to care that much about data access patterns but modelling data so it fits nicely into DynamoDB access patterns are the first thing they have to think about. It has been my observation that developers tend not to like it.
If you want to know more I suggest that you watch this year’s DAT401 session on Youtube once it is available (DAT401 – Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB).
– Amazon Comprehend now understand medical text
– A new service AWS Elemental MediaConnect for video ingestion and distribution
Day 3 will be huge since Andy Jassy’s keynote is in the morning and it will be packed with updates.
Now that Re:Invent is at full speed the flurry of new features is relentless. Let’s go through a couple of the most noteworthy announcements from Day 1.
IoT has received a lot of love.
- IoT sitewise (preview) is targeting entire plants and industrial equipment instead of small sensors normally associated with IoT.
- IoT events (preview) is targeted for event correlation between multiple sensors and helps to recognise system-wide events and also enables alerting in such occurrences.
- IoT greengrass is extended with external app-connectors, hardware root of trust (using Hardware Security Modules or Trusted Platform Modules) and more.
- IoT Things Graph (preview) is an easy way for developers to build IoT applications. IoT Things Graph hides low level details and enables packaging as reusable components.
- Also, Bluetooth Low Energy is now supported in Amazon RTOS (beta).
So overall there were quite a few announcements in the IoT space. If you are doing IoT there should some interesting features announced which makes life a lot easier.
AWS Transit gateway
A new feature which allows users to connect their VPC’s and on-premise networks to a single gateway. Transit gateway acts as a centralised hub where VPC’s and on-premise connect as spokes. It includes support for dynamic and static routing. Since Transit gateway allows forwarding of DNS queries it is possible to resolve IP’s on other VPC’s that are connected to Transit gateway. In addition, there are monitoring, security and management using IAM and Cloudwatch. There’s also support for Equal Cost Multipath (ECMP) when routing via VPN connections to on-premise.
Overall Transit gateway is a huge step forward in networking. It makes creating complex topologies much easier. Especially enterprise-customers who might have multiple accounts used by multiple departments should now be able to create more uniform access to on-premise instead of connecting different VPC’s individually via VPN/Direct Connect.
AWS Global Accelerator
If Transit gateway is useful for inter-VPC communications then AWS Global Accelerator is at least equally useful but targeted to the Internet. With Global Accelerator, applications can make use of the AWS global networking backbone. Global accelerator removes the need for managing different IP-addresses for different regions. Global Accelerator reserves 2 IP’s and anycasts on those globally. Traffic is directed to the AWS network in the nearest POP and from there it travels via the AWS network until it reaches its endpoint. Endpoints can be configured as different AZ’s or regions and are continuously health-checked. Global Accelerator greatly simplifies multi-region setups and provides smoother end-user experience.
This is definitely on my “gotta try it out”-list. One more step in making multi-region setups more common.
Nitros and more
With the new AWS hypervisor system called “Nitro” there is now a new instance type C5n featuring 100Gbps networking speed. Not much more about that can be said. More bandwidth is always good and for customers who are maxing out 10Gbps or 25Gbps this is a welcome relief.
Then we have a very interesting announcement. EC2 A1 instances. The interesting part is the 64bit ARM-processor with custom designed silicon called “Graviton”. That’s it – no x86. There are several Linux-distributions which can be run on these instances and it will intresting to see what kind of adaptation these machines will receive. Moving out of the AWS context it’s also interesting to see ARM-processors starting to take on areas normally dominated by x86 chips. Apple’s A10 chip and now Graviton from Amazon. Should Intel feel threatened? Time will tell.
Ever wondered what kind of server-fleet is running customers lambda-functions? Or Fargate-containers? Wonder no more since AWS has released “Firecracker” which is microVM for a running container. Will this technology find its way into other open-source projects?
Today’s announcements have been touching some very fundamental building blocks. Fundamentals have changed so much that developing multi-region applications or multi-account networking look a lot different than they did 24h ago.
More announcements and news are being released throughout the week. I’ll post again tomorrow tomorrow; let’s see what suprises AWS has prepared for us!
Before diving into the technical aspects and the new announcements I’ll take a moment and write a bit about the time before the actual conference. If you have never participated before there are a couple of ‘gotchas’.
Travelling from Europe is tiring and it’s better to arrive early to give yourself time to recuperate. Also when travelling from Europe remember that if you have a connecting flight inside the USA you will have to do the customs/CBP stuff when you first land. This combined with the fact that your luggage must be collected from baggage claim and re-checked into the domestic US-flight means that you should reserve enough time for your connection, otherwise you will experience added stress and potentially miss your connection.
When in Las Vegas…
Remember that the whole city is designed specifically to separate people from the contents of their wallets. Everything costs and more often than not the price is not cheap. Las Vegas is in the middle of the desert and the air is dry, this is something you should take into account if you have sensitive skin. For me, the effect of the dry air is best visible on my beard. In Finland, it is usually much more curly due to more humid air. Here it straightens out considerably. I bet you wanted to know that 🙂
Las Vegas Boulevard aka ‘The strip’ isn’t that long on the map but it is long enough that moving between different venues takes time. If at all possible try to plan your schedule so that you minimize moving between venues. AWS has booked shuttle buses, there’s a monorail and you can walk but all the options take time and most of the time there will be a sea of other attendees moving in addition to you.
Also, contact other companies and people. There is a huge amount of different smaller gatherings and parties organized by different companies. The opportunity to network and to get to know people is huge. Attending your local AWS meetups will help you connect with others.
In the end, the conferences usually are best experienced first hand. The technical information can be learned from streaming videos and Youtube-videos. Being visible and networking is something that won’t be possible if you don’t attend. Furthermore, attending with only one person is overwhelming. Absorbing everything that is available is a huge task. Combined with networking and possibly having a booth is even more overwhelming. Consider sending more people, preferably 2-3 and if you have clients with you or are having a stand in the expo you need even more. Naturally, for a consulting business, this might be a pretty big investment. There are the costs of the trip itself (tickets, flights, hotels, per diem, etc) but in addition to these, the attendees are not doing billable work. So attending Re:Invent can also be seen as a commitment, you are committed to your partnership with AWS.
Actual announcements and news!
On Sunday, 25th of November the actual Re:Invent hasn’t started yet, however, there are some program items already on Sunday, more specifically the Midnight Madness and Tatonka challenge. Midnight Madness is a launch party or pre-party and Tatonka challenge is an event where attendees try to eat huge quantities of chicken wings. I had the advantage that I live in Tampere which is the wing-capital of Finland. Long story short: I didn’t win Tatonka but in addition to Tatonka and Midnight Madness there was the first official launch: AWS announced ‘AWS Robomaker’.
Robomaker is intended to help developers creating robotic applications. AWS has an extended open-source framework “ROS” and included extensions so it includes connectivity to the cloud. Robomaker aims to be a complete development environment and includes an integrated development environment, simulation possibilities and fleet management.
Robotics is not an area which would come up in my daily work. However, if you are working in such field this new offering might be useful for you. I also hope that offerings like Robomaker will help different ecosystems to grow. Making robotics and robot-development accessible to a bigger audience will help innovation and might produce completely new products and offerings.
In addition to Robomaker, there were also some interesting announcements and new features that were published during the last few days. However, these might go unnoticed on the grand scale of Re:Invent. Here are some of the new features sampled by me (my listing is not comprehensive)
EFS infrequent storage class – coming soon. EFS will be getting an infrequent storage class much like S3 has. Naturally, this helps with cost-control and should be interesting to anyone using EFS.
- Amazon Rekognition. Improved facial analysis, detects faces with greater accuracy and confidence. Should be interesting if your use case includes Amazon Rekognition.
- AWS DataSync. New service to automate transferring data between on-premises storage and S3 or EFS. This service is mostly aimed at hybrid solutions and cloud migrations. Definitely, something to check out if you are working in the hybrid/migration space.
- S3 batch operations – preview. Simplify the management of huge amounts of objects. Bulk operations are usually custom code, developed by AWS clients themselves. Batch operations aim to reduce the complexity that bulk operations usually require. Moving objects, replacing tags or managing access controls. Use cases are almost limitless ranging from compliance to backups to data migrations.
That’s it for Sunday in Vegas. Let’s see what Monday brings!