Transistion to Cloud - why some companies are struggling to implement this change?

At AppGenie we utilise a number of different Cloud-based platforms including AWS, Azure, Salesforce, GCP and AliCloud. We have significant experience within each of the Function as a Service (FaaS) offerings from these platforms. Our own Salesforce ISV/AppExchange products FTP2SF is built leveraging AWS Lambda along with some ancillary Platform as a Service (PaaS) offerings. This article explains the benefits of transitioning to cloud with a serverless implementation and how it can help your business be successful. It is a roadmap to your cloud success.

Our learnings

Our learnings from this environment are considerable and while the transition to cloud and FaaS has been sometimes technically challenging it has rewarded us with a number of benefits:

• The performance of our applications is practically straight-line with minimal performance differential measurable over the number of connections we can scale to;
• Availability has never been an issue with the ability to deploy multiple run time environments allowing us to scale dev and test deployments directly in Production releases;
• Development time is significantly reduced, measuring in weeks what would normally have taken months;
• Cost savings have been measured at around 70% versus a more traditional infrastructure stack.
• No lock in … this has been a huge reward, we know FTP2SF will work irrespective of what FaaS provider we choose to utilise and more importantly we transition between providers in days not months.

Given the above I am often surprised at the reluctance for customers to transition away from the Virtual Machine (VM) lift and shift model and empirically there are a number of Enterprise customers that are rejecting Cloud platforms and transitioning back to On-Prem and Hybrid hosting solutions in preference due to cost.

So why is this so difficult?

As an enterprise architect, I've had many discussions around why this has proven to be so difficult for so many businesses, from SME to Enterprise, Start-up to Mom & Pop. The benefits of serverless technology outweigh both on-prem and hosted cloud solutions. So why are people not adopting serverless?

• Lack of understanding People do not understand exactly how systems will work together or be replaced;
• Cost People do not understand costs which will be involved, both to implement and from an on=going perspective;
• Time People do not understand how long the process will take, and are confused by what needs to happen when;
• Success People do not feel confident that they will be successful, so they stick with older technology that they have known;;
• Change People are not comfortable to move into new technology, there is a fear of change;
• Risk People do not understand the risk elements with their transition, particular for Production.

So why have we been successful and others haven’t?

We have invested time and money into this process and we have become experts, here to guide you. I believe the single biggest factor involved is that we own the development and delivery of our own software. We made a commitment early to the FaaS model and have adopted this in our entire development and delivery process. If we can deliver something using FaaS, we will. This ability to drive the architecture in the direction we choose is critical to the success of transition to Severless.

So what can you do if you are a business that only uses Componentised Off-The-Shelf (COTS) Products and are locked into a 2 or 3 Tier architecture? How do you benefit from the Serverless infrastructure and keep a handle on costs?

Dev and Test Environments

Most businesses immediately identify Dev and Test environments as being the most likely to transition to cloud and provide a cost benefit, this is due to the non-critical nature of these systems. Basically you can shut the stack down after hours and power-up at a predetermined time or manually. We take this approach also when we test our platforms, as there is no point running something we (or our customers) don’t need.

So now we are left with the same problem most people come to us with – Now what do we do with Production?

Serverless Production Environment - IDAM

My first recommendation is to separate Identity Management and Access control and utilise your favourite IDAM platform to support this – most often this is deployed in Azure.

Your first step is to maintain control of the identities of users and "system" user accounts, without a centralised IDAM strategy your ability to configure the environments and services that are required will incur significant delays, you DON'T want to be running multiple IDAM platforms, you DO want to be running SSO across your application stack. Here's our recommendations for this space:

• Azure Active Directory: Cloud-based identity and access management service for managing user authentication and authorization to resources;
• Microsoft System Center Configuration Manager (SCCM): Manage the deployment and configuration of devices, software, and updates in an organization's network;
• Microsoft BitLocker: Full-disk encryption tool for protecting data on Windows devices;
• Microsoft AppLocker: Control the applications that can run on a Windows device;
• Microsoft Defender for Endpoint: Unified endpoint security platform for protecting devices from cyber threats;
• Azure Sentinel: Cloud-based security information and event management (SIEM) service for threat detection, investigation, and response;
• Microsoft 365 Defender: Security suite for integrated threat protection, security management, and compliance capabilities across Microsoft 365 services.

Serverless Production Environment – API Management

Once you have transitioned your IDAM functions, next steps are to make your life easier from an integration perspective. Most Enterprise customers have multiple messaging platforms and integration patterns – so next step is to get control over your APIs this allows the same configuration, access control and security controls as your IDAM platform across your API landscape, choose any one of the following:

• Azure API Management: Azure cloud based service to publish, monitor, and manage APIs. It supports versioning, access control using Azure AD, and documentation;
• MuleSoft Anypoint Platform: Mulesoft’s API management platform that enables you to design, build, deploy, and manage APIs. It supports versioning, access control using OAuth2 and other authentication protocols, and documentation generation. You can also use it to monitor API usage and performance, automate API testing, and enforce security policies;
• Amazon API Gateway: AWS’s fully managed service that allows you to create, publish, and manage APIs. It supports versioning, access control using AWS IAM, and documentation generation. You can also use it to monitor API usage and performance, and to enforce throttling and caching policies;
• Kong: If you’re into open-source Kong provides an API gateway and management platform that allows you to create, deploy, and manage APIs. It supports versioning, access control using OAuth2 and other authentication protocols, and documentation generation. You can also use it to monitor API usage and performance, and to enforce security policies.

Note that without API controls in place, the risk of exposing data through non prod APIs and misconfiguration is exponentially greater. The potential brand damage is compounded by the personal liability that directors hold based on Section 180 of the Companies Act along with the Privacy Act 1988 and the Notifiable Data Breaches scheme.

Serverless Production Environment – Integration

My preference for integration leans towards toolsets from AWS, the Serverless capabilities that we utilise most often include Salesforce components and with the introduction of Hyperforce and the roadmap that is to be delivered we can’t go past the performance of EventBridge:

• Microsoft Integration Platform: The platform includes Azure Logic Apps, Azure Service Bus, Azure Event Grid, Azure API Management, and Azure Functions, among others. These services enable you to build scalable and secure integration solutions that can automate workflows, process data, and connect applications across your organization. Additionally, the platform includes robust monitoring and management capabilities, as well as integration with Azure Active Directory for access control;
• AWS Integration Platform: The platform includes AWS AppSync, AWS Step Functions, Amazon EventBridge, Amazon MQ, AWS Glue and others. These services allow you to build scalable and reliable integration solutions that can process data, automate workflows, and connect applications in real-time. Additionally, the platform includes features for monitoring and managing your integrations, as well as integration with AWS Identity and Access Management (IAM) for access control.

Now you're going to tell me you already have any one of a number of ETL tools that a developer told you was the cheapest and best toolset under the sun ... Stop padding their CV, use any of the above, get rid of everything else its just noise.

Serverless Production Environment – Data

Data access is the next problem to be solved, this one is usually the most complex as you will want to support all of your Database versions and types while still minimising these. In this space most businesses end up either in the OpenSource model using MySQL and PostgreSQL or at the top end of town with Oracle Databases, along with the SQL Server database that no one ever remembers about:

• AWS Relational Database Service (RDS): The platform includes Azure Logic Apps, Azure Service Bus, Azure Event Grid, Azure API Management, and Azure Functions, among others. These services enable you to build scalable and secure integration solutions that can automate workflows, process data, and connect applications across your organization. Additionally, the platform includes robust monitoring and management capabilities, as well as integration with Azure Active Directory for access control;
• Microsoft Azure SQL Database: The platform includes Azure Logic Apps, Azure Service Bus, Azure Event Grid, Azure API Management, and Azure Functions, among others. These services enable you to build scalable and secure integration solutions that can automate workflows, process data, and connect applications across your organization. Additionally, the platform includes robust monitoring and management capabilities, as well as integration with Azure Active Directory for access control;
• MongoDB Atlas: The platform includes Azure Logic Apps, Azure Service Bus, Azure Event Grid, Azure API Management, and Azure Functions, among others. These services enable you to build scalable and secure integration solutions that can automate workflows, process data, and connect applications across your organization. Additionally, the platform includes robust monitoring and management capabilities, as well as integration with Azure Active Directory for access control;
• Google Cloud SQL: The platform includes Azure Logic Apps, Azure Service Bus, Azure Event Grid, Azure API Management, and Azure Functions, among others. These services enable you to build scalable and secure integration solutions that can automate workflows, process data, and connect applications across your organization. Additionally, the platform includes robust monitoring and management capabilities, as well as integration with Azure Active Directory for access control;
• Oracle Cloud Infrastructure Databas: The platform includes Azure Logic Apps, Azure Service Bus, Azure Event Grid, Azure API Management, and Azure Functions, among others. These services enable you to build scalable and secure integration solutions that can automate workflows, process data, and connect applications across your organization. Additionally, the platform includes robust monitoring and management capabilities, as well as integration with Azure Active Directory for access control;

Now you have your data all in the same "location", you can run any reporting engine or COTS App across the top of them. If you want to run all of them you can, but realistically you want to try and minimise your footprint, you can even run your Snowflake instance consuming events from all of them that way you have your Data Warehousing solution available along with your Messaging sorted and you can even throw in that you now have ISO20022 certified platforms.

Serverless Production Environment – Checkpoint

At this point you have transition Backend Data access, Identity management and Security platforms to a Severless or as a Service Environment. no application has been shifted from your On-Prem or Private Cloud solution. The important note is that this rollout of services can happen without impact to any specific application - even your Data acceess can be transitioned without impact.

Serverless Production Environment – Applications

The greatest cost component of most public clouds and therefore the point where most transition to cloud programs fail, is when “risk" is mitigated by introducing the lift and shift of a physical machine to the equivalent virtual instance running in the cloud. You are charged for every second your virtual machine is running. This is the single biggest reason for failure of most transition to cloud projects. Using one of the following reduces your expense by only paying for what you use:

• AWS Fargate: Fargate is a container-based compute engine that allows users to run long-running or persistent workloads such as databases, web servers, or batch processing. It provides automatic scaling, pay-as-you-go pricing, and easy integration with other AWS services;
• Kubernetes: Kubernetes is a popular open-source container orchestration system that allows users to deploy and manage containerized applications on a large scale. Kubernetes provides a flexible and scalable platform for container-based compute, with support for a wide range of container runtimes and platforms;
• Google Cloud Run: Google Cloud Run is a fully managed serverless compute platform for containerized applications that supports both stateless and stateful workloads. It provides automatic scaling, pay-per-use pricing, and seamless integration with other Google Cloud services.;
• Microsoft Azure Container Instances: Microsoft Azure Container Instances is a serverless container service that allows users to run containers without managing any infrastructure. It provides rapid deployment, auto-scaling, and per-second billing;
• Alibaba Cloud Container Service: Alibaba Cloud Container Service is a container orchestration service that supports both Kubernetes and Docker Swarm. It provides automatic scaling, high availability, and easy integration with other Alibaba Cloud services.

Production as a Service

Now that your cloud transition has been completed you are now using a fully as a Service model:

• Software as a Service
• Function as a Service
• Database as a Service
• No Virtual Machines
• Your costs are 100% usage based
• Costs are pared sown to minimal network overheads
• no vendor or component dependancies

Using your integrated IDAM & API Management layers, you now have the ability to transition your containers to different providers based on the best price performance model - in most cases the containers are interchangable and your configuration of these containers will be committed into your GIT prod repo. On top of this the Container model allows you to identify the cost centres associated with usage, you now have a usage based charge back model that can support a per user charge.

Feel free to reach out to me (mark at appgenie) anytime to discuss Serverless Computing and we can save your organisation money while increasing performance, reliability and security.