How did you make your transition to the cloud? What worked and what didn't work?

Cloud - At Palantir, we moved all on prem servers to the cloud (AWS) but did it all as code (as opposed to configuration). There are several reasons for doing it this way and this approach while not new is recently gaining more traction. The idea here is don’t configure anything in your infra, instead describe the end state you want using a declarative language like terraform and then let the tools build the infrastructure for you. This is contrary to the classical approach where it is a combination of code, interspersed with manual config, resulting in fragile infrastructure.
By using this approach, we:
  1. are somewhat cloud agnostic (as to the infra is described in a language that works with all major cloud providers)
  2. don’t have any administrator as a single point of failure
  3. have infra that is repeatable and consistent across environments
  4. have a process that is fully integrated it with VCS, CI/CD so there is continuous deployment and automatic tracking of all changes

17 comments

#cloud,#transition,#security,#aws https://www.pulse.qa

Pulse User

At Palantir, we moved all on prem servers to the cloud (AWS) but did it all as code (as opposed to configuration). There are several reasons for doing it this way and this approach while not new is recently gaining more traction. The idea here is don’t configure anything in your infra, instead describe the end state you want using a declarative language like terraform and then let the tools build the infrastructure for you. This is contrary to the classical approach where it is a combination of code, interspersed with manual config, resulting in fragile infrastructure.
By using this approach, we:

  1. are somewhat cloud agnostic (as to the infra is described in a language that works with all major cloud providers)
  2. don’t have any administrator as a single point of failure
  3. have infra that is repeatable and consistent across environments
  4. have a process that is fully integrated it with VCS, CI/CD so there is continuous deployment and automatic tracking of all changes

Pulse User

Infrastructure as Code should be table stakes for any and all Cloud migrations/initiatives. In addition to being more #cloud agnostic, it helps guard against the classic problems around configuration drift and repeatability of deployments. It also helps with security challenges as you can define your policies as code as well. Having said that, to avoid the "Bob/Mary changed something in production outside of a code update problem", don't allow direct user login to your cloud infrastructure. (or at the very least, have an audit log trail)

Pulse User

we moved existing equipment into a physical hosting environment then migrated to cloud services (IaaS) as infrastructure aged or became unsuitable (over a 24 month period). We also identified systems for a move to PaaS where ever possible, and establish services such as Azure AD very early in the process

Pulse User

I Still suggest Owning server, somewhere in USA. Good configuration of Server cost nearly $2k, bandwidth somewhere near $2/Mbps and rack space rent. That way you have your own cloud. We always do this.

Pulse User

When we chose to go fully cloud based we actually completely reworked our release strategy and codebase. We cut up features and modules into modular applications which would run seperately but would be used as a JSON API by the mainframe. This allowed for dedicated devs per module and a staged release strategy causing high costs to start but quickly generating a return on investment by providing high scalability and backwards compatibility.

Pulse User

Krunal, not to be disparaging, but renting racks in a colo and populating them with N servers and X Mbps of bandwidth is extremely far from a (private) Cloud. There is nothing elastic or composable about that architecture/solution to start with.

Pulse User

We started off in cloud itself as we don't have permanent offices. Further the skill set to manage servers and ensure proper back up is taken was missing. Hence all applications are outsourced and hosted on cloud

Pulse User

When you decide to make the transition to code from on-prem the first thing you realize is that auto-scalability, IAC (Infra as Configuration) are much easier to implement than it would have hitherto been possible.

At the same time, we realize that with power comes responsibility. While it is easy to stand up new servers and automate stuff, it is also equally easy to let these scripts run amok and start creating redundant resources that would cost you a bomb eventually. Hence it is extremely important to strategize this. When am I going to create new servers using IAC? What would be my auto scaling strategy? etc. are questions that you need to ask.

Infra architecture is something that you need to care about whether you deploy stuff to the cloud or on-prem. Hence I would not emphasize the need for infra architecture as a distinguishing feature of what changes when you take your infra to the cloud. But the infra architecture needs to be solid and needs to form the basis of the declarative language that describes your target architecture. We have increasingly sophisticated tools like Terraform, K8S etc. that consume this declarative language and help create and maintain these environments. These (at least for now) work better in the cloud than on-prem.

Also, the people required to maintain infra differ between on-prem and cloud.

Finally, is cloud mobility a concern for you? Do you in theory need to write IAC scripts that can switch between AWS or GCP or AZURE? I am ambivalent on this. By focusing too much on cloud mobility, you might be compromising on the feature set that you intend to use from your current cloud provider. Stuff like auto scaling, security groups, sub netting strategy need to still be customized in accordance with your cloud provider.

Pulse User

When starting the road to the Cloud the main engine to address the costs and efforts should be a clear business model, where the adoption of the Cloud is a mechanism to increase productivity. You go to the Cloud to get business results.

In Patagonia IT (www.patagonia-it.com), we started to deploy our own services on these platforms, and becoming more competitives delivering services faster an with better SLAs than anyone.

We decided to give that value to our Software Development Services clients and started a road to be AWS partners.

Now we have been supporting the design of the cloud serverless architectures for Airlines, Government Institutions and lots of companies, and the main focus is always the same: Faster time to market, scalability and freeing strategic resources of our clients to put them in their main business challenges.

What worked? Every person of the team involved in the design of the solutions, not only system engineers, developers too.

What doesn't worked? Try to do the same we do in on-premises architectures in serverless architectures, you need to rethink all using the new Cloud tools

Pulse User

Differently from many other organizations, Royal started it’s cloud journey by hosting new applications that required elasticity and a faster time to market. We didn’t perform any datacenter migration and this is in our radar probably for 2020 for the apps that make sense. All of this hype of emptying traditional datacenters are not justified when you have huge infrastructure investments, applications that don’t require on demand capacity and a financial model that prefers capex over opex. We don’t follow trends, we do what is right and makes sense.

Pulse User

I'm always in front and utilizing or innovating future-destined technology and when I embraced cloud in 2009 I was early and had to build a lot of the infrastructure that existed in traditional lT environments, like DNS, CNAMES, etc. but it was the speed of provisioning and low cost that fueled the eventual for granted adoption. So new opportunities were first because they offered immediate payoff and by thinking forward set up a field of technology and practices that emerged as patterns. And patterns can replicate, evolve, and mature. I have always mentored a "design" principal over "implementation" where possible. This allows a fluid evolution of the business with progress instead of reimplementing every tech generation which turnover is faster and faster. For clients, it is all over the spectrum and no two are alike, but common in many ways, more than meets the outside eye when it comes to the foundation of technology due to the open source movement and consolidation of core enterprise solutions. But the processes and IP are night and day. Each implementing organic and strategic initiatives and methodologies. Some begin with the least vulnerable computing assets to gain experience and confidence, others lift and shift faster that a flock of birds flying south for winter, while others apply a scalable methodology of analysis and decomposition to identify friction and dependencies then prioritize, based upon a set of established goals--cost, scale, speed to market, etc. and spin up concurrent and independent teams. It is important to understand that not only is the infrastructure impacted but perhaps more important the human capital is impacted as well and it is wise to address this facet of change first. Fear will manifest and management must mitigate and train the human element to embrace a new way of accomplishing the business goals one is tasked with. I have taken 100-year old .NET shops, AS400, glasshouses to the cloud and tackling he human element was the greatest challenge. But done right delivers a big implicit payback that is difficult to measure like the cost of cloud computing--which is not always easy and another point to master before it destroys your flowering cloud garden. Insight and cost control is critical and yields greater flexibility and ability to scale up and down in and out as needed and done strategically via prediction analytics. The cloud is not suitable to traditional agile process flows and facilitates a collaborative ownership model were a continuous and fluid methodology in harnessed providing the business with a continuous collaborative independent operating model that removes the traditional process blocks experienced in the sequential roadmap driven approach. Netflix is the poster child for this culture, and Pivotal adopted many NetflixOSS components, which are special-built for an always-on, aways-evolving, always experimenting, and always exercising breakage to prove resilience. It would seem the personality of the driving stakeholder will set the stage for how a companies move to the cloud will play out. Cautious will go slow and value risk control, the Git'r done will want to move fast and value process control. The rest are somewhere in between and a composite of varying degrees. Hopes this helps.

Pulse User

I had the opportunity to take hotelconnections through its 2017 AWS Cloud migration and here are some learning. I thought it was easy lift and shift but after the fact realized, there are many facets to consider the migration. Following are items in order to observe carefully.
1. Currency of your current infrastructure and software is a liability
If your infrastructure and software is not upto current standard it puts a huge migration cost or even limits some of the capabilities which you can gain, so definitely know your current infra/software currency and gaps. Will add $$ into your overall migration. Learnt this the hard way where we had several components in older version which had to be updated prior to start.
2. Consider your sizing of infra
Usually with cloud you are able to reduce your footprint as long as you have the right redundancy, we were able to reduce the number of servers by about 10% and also add redundancy at the same time.
3. Security for your new infrastructure
Cloud comes with security to certain degree, however unless you have strong SOC or NOC, engaging a partner helps ensure the right separation and controls are built, and monitoring is installed. Setting the right group security for public and private components are important. Leveraged a managed service partner who did a great job. (@hosting.com)
4. Bring your team along
You need your core tech teams along with you, share the benefits and ensure that no one’s job is being taken away by a vendor and there is lot more for the team. This will pay back immensely when you hit the road bumps during migration. My team helped me immensely when we got stuck.
5. Business Case to justify the migration
Buy in from your C-exec , and peer team is very important, as migration will involve downtime, and champion the cause and its benefits across the entire organization. Each peer leader rallied for the case when we had to take a migration downtime of 4hrs.
The above info is available at https://www.linkedin.com/pulse/embracing-cloud-plan-surprises-july-4th-2017-sreenivas-sreenath/

Pulse User

Like a lot of topics - just because you have a hammer everything is not a nail... Having said that the comments here about the need to change your approaches and thinking are spot on. Infrastructure as code and using concepts like containerization is critical to success. Using traditional thinking in the cloud can get very expensive in a hurry! Make the move to Saas first, then offload other compute loads in an elastic model.

Pulse User

Cloud is a mean to an end. We must determine what business goals we want to drive towards, before investing in a cloud journey. From our 17+ Enterprise cloud migrations in last few years, we have learned many pitfalls in this journey. Cloud transformation is not a one size fits for all solution. Business needs and long-term business horizons are very different among enterprises. Here is what we have learned: • Re-hosting apps and infra to Cloud IaaS will not yield much business value if there is a predictable load and lifespan for business apps. • Agility requirements vary significantly across enterprises. So, investing time and money for the kool Aids like ultra-speed provisioning automation (Infra as a code) may not be needed everywhere. If we don’t assess the real need and payback time for such investment, we will end up with another complex ‘system’ to babysit and manage. So, it should be a carefully evaluated decision. • App modernization should be the primary driver for Cloud adoption. And the modernization journey should be chosen carefully, to show incremental business value. • We use our ‘3D’ model to help our customers in their journey. First, we identify strained system within customer infra real estate and ‘De-Stress’ them. The solution could a lift and shift move to public cloud to alleviate issues like Reliability, scalability or Security. We could solve these issues with re-hosting, API Encapsulation or even replacing them with a third-party SaaS. The Second stage is our next ‘D’ where we do “De-Clutter” of legacy systems to make them extendible and maintainable. We may refactor or Re-platform the apps to make them ready for digital enhancements. And finally, we get to the third D: “Di-gitalize”, where the apps will be rewritten for new digital capabilities. This must be driven by organization’s new identified business horizons. • All these transformation phases will be continuously measured against Purpose, Effects, Costs and Risks using our ‘Business Affinity Matrix’ so that business folks are happy.

Pulse User

Cloud transition form on premises servers was quite easy. The configuration of DB and apps layers were quick.

Pulse User

The cloud journey requires the IT strategic plan goals to align with the overall business strategic plan business outcomes.  Think cloud first for hosting new internal or customer facing solutions and products, you may not ultimately choose the cloud for everything, but more importantly you will choose the cloud where its the best fit.     

Pulse User

An Organization can start its Transition to the cloud with its Satellite Non-core applications for e.g. CRM is a very clear example of a pure SaaS play cloud app.  This app will integrate with your Core On-Premise ERP or legacy systems.  Cloud based HRM is also another example of SaaS.   If there exists home grown/purchased Applications on Java or .Net or any open source then IaaS can be attempted by hosting the App on a Cloud based Virtual Server instead of hosting it in an On-Premise server.  Once your confidence and comfort levels are established, you can move into larger cloud applications.  Most organizations work on a hybrid cloud model.  Make sure you prepare your TCO well in advance before embarking into the cloud.