What you should know about the WSO2 Platform Migration

All the WSO2 products — WSO2 API Manager, WSO2 Micro-integrator and WSO2 Identity Server rely on 02 main components:

• Database(s) — which stores all the metadata related to the product and the published APIs, integrations, Users, etc.
• File System — which stores product binaries and configuration files

WSO2, as a middleware product engineering vendor, releases at least one major version and 1–2 minor versions of their products every year with a lot of new features and enhancements. As a result, WSO2 strongly recommend to upgrade/ migrate their platform to the latest version at least every 3 years.

A study indicates that majority of customers starts the WSO2 platform migration without a proper planning and they end up facing lot of challenges resulting long-running migration projects, increased infrastructure cost, increased services cost and impacting day-to-day business.

This post comes to you as a good planning guide/ eye-opener, so you can plan the WSO2 platform migration well and reduce the challenges you may face in the future.

Types of migrations

First, let's look at why you want to migrate. Below are the main reasons/ types of migration we come across:

• Infrastructure migration — this happens when we want to move from one infrastructure platform to another (ex: On-premise to Cloud/ VMs to Containers/ Kubernetes).
• Database vendor migration — this happens when we want to move from one database vendor to another (ex: MySQL to MSSQL/ MSSQL to Oracle). WSO2 does not provide any tools/ support on this as there are differences between database vendors. A 3rd party tool or a tool provided by the database vendors can help in this use case to read from source database(s) and write into destination database(s).
• Deployment architecture changes — this happens when we want to move from one deployment pattern to another (ex: All-in-one to Distributed). This will be done by adding more instances and segregating product profiles among them.
• 3rd party components migration — this happens when the WSO2 platform is integrated with other 3rd party components and they need to be migrated to the latest version. If the integration is done via standard interfaces, then this will be very straightforward. If the integration is done via some custom interfaces or customizations, then the WSO2 products needs to be tested for the new version of the 3rd party component.
• WSO2 product version (major or minor) migration — this happens when we want to migrate between minor or major versions (ex: minor — APIM 4.0.0 to APIM 4.1.0/ major — APIM 3.0.0 to APIM 4.1.0). Updates between minor versions or update levels can be done seamlessly using WSO2 Update 2.0 tool. Migrating between major version will be a bit complex as there may be lot of changes in the product architecture and product APIs.

When should you migrate?

We need to carefully analyse the scenarios/ reasons for migrating to the latest version. Below are some of the example scenarios you can consider for migrating to the latest platform:

• When the current version reaches the end-of-life (EOL) support — every WSO2 product has a EOL and it is listed in the WSO2 Support Matrix page. Typically, it’s 3+1 years from the date of release. We need to upgrade the current version before we reach EOL to avoid support complications.

WSO2 Product Support Lifecycle | Source: WSO2

• Leverage new features of the newer version — by comparing the current version and the latest version, we can identify what are the key features/ benefits that will add value to the business. If there are significant value which we can bring into the business via the new features, it’s recommended to migrate to the latest version no matter when is the EOL of the current version.

What changes users can expect with a WSO2 product version upgrade?

With a WSO2 product version upgrade, users can expect changes in the below areas:

• Feature changes — new features can be added, existing features can be enhanced/ improved, existing features can be deprecated and existing features can be removed during a product version upgrade. It’s always advised to check the “What has changed” documentation link to understand differences between specific product versions.
• Database changes — version upgrades (specially the major versions) will have changes in both database schema and data. The database migration will be done using a set of scripts provided by WSO2.
• Customization changes — if we have done any customizations in the current version, it’s better to analyze and see whether we can map them into out-of-the-box features in the newer version. If yes, then we can remove those customizations after the migration. If no, we will have to migrate those customizations to support the newer version. While WSO2 is a extensible platform, my recommendation is to avoid any customizations as they are expensive to maintain/ support.
• Configuration management changes — older version of WSO2 products use XML-based configuration and it is done in multiple locations in the file system. The newer versions use TOML-based configuration and it is centralised using a single configuration file. If you are upgrading from an older version to a newer version, the equivalent configuration needs to be done in the newer version.

Factors to consider when planning your migration

Below are some of the components which we need to analyze in detail when planning the migration:

• No of APIs/ Integrations/ Users — when the no of APIs/ Integrations/ Users is higher, the complexity and time it takes to migrate will also be affected. WSO2 provides a set of tools/ scripts to import and export these artifacts between environments and they are available across all different version upgrade scenarios/ combinations.
• No of Applications/ Subscriptions — with API Manager, the number of Applications and Subscriptions play a significant role when migrating to the newer version. Retaining them will avoid recreating them. Recreating applications and subscribing to them will be very a tedious task if the no of APIs and Consumers are high.
• No of Users/ Roles/ Access Tokens — if the no of users/ roles and access tokens are high, we need to have a mechanism to migrate them over to the new environment via scripts. WSO2 provides a set of scripts to seamlessly migrate them across different versions.

Migration Process

Below is the standard migration process every WSO2 product migration should follow. We can start with the lower environment(s) and progress through the production environment. This will be an iterative approach where we will go through the process until we successfully migrate all the environment(s).

Standard Migration Process

• Configuration migration — as WSO2 older product versions use a segregated XML-based configuration model and the newer versions use a centralized TOML-based configuration model, both has to be mapped. We need to compare the existing configuration files with a vanilla/ base version of the configuration file and identify the configuration changes which needs to be applied in the target version.
• Customizations migration — WSO2 products are fully extensible and it allows us to extend the functionality by applying customizations. Some customizations such as UI changes needs to be manually migrated/ recreated as every major version uses an enhanced version of the UI theming engine. For customizations such as custom mediators, custom handlers such as custom OAuth grant types and custom key manager connectors, etc. the source code has to be recompiled with the proper dependency versions and validated against the target version.
• Database(s) migration — the number of databases need to be migrated will depend on the WSO2 product. The general approach of database migration is:
  [1] Take a backup of the exiting database(s)
  [2] Set the exiting environment read-only (if we want to achieve zero downtime and data conflicts at the same time)
  [3] Run the appropriate database scripts provided by WSO2 depending on the database vendor. WSO2 provides different database scripts to migrate databases as the schema and the datatypes will vary depending on the database vendor.
  [4] Review the logs, and if there are any errors in the migration, restore the previous database and rerun the scripts until all the issues are resolved.

Ready for the final step? — Production Migration

Now that we have gone through the migration prerequisites, migration process, etc. it’s time to discuss what we need to consider about the production environment migration. Below are some of the key aspects you need to consider:

• Have we done the lower environment migrations successfully? — if not, lets fix any remaining issues and make sure alle the lower environments are successfully migrated.
• Are you satisfied with the performance and features? — all the new releases come with some performance improvements and new features. It’s always recommended to conduct a performance test in your lower environments to confirm that there are no performance bottlenecks and all the new features are working as expected.
• Do you have a contingency plan? — anything can go wrong in the last minute. It’s always better to have a contingency plan so we can rollback to the previous version in case if something goes wrong so business will not be impacted.
• Is your checklist/ runbook accurate? — all the migration steps including configuration migration, customization migration and database migration are unique to every customer and all of them needs to be well documented. Validate the migration checklist/ runbook itself to see whether we have captured everything accurately. This will be your how-to guide/ knowledgebase.
• Is automation in place? — there are plenty of areas you can bring automation into the exercise and it will reduce a lot of time. Evaluate where you can bring automation and reduce time it takes and it will increase both productivity and accuracy avoiding manual interventions.
• Identify the migration window — this is very important for the production migration. Identify the off-work/ off-peak period and determine the migration window to perform the final migration and traffic redirection.
• Do we have the test cases ready? — it’s always recommended to have a set of test cases for each environment for the final validation. Automating them is even better. Make sure we run those test cases and validate each environment before handing over to users.
• Finally, have you contacted/ informed WSO2? — it’s always recommended to initiate a conversation with the WSO2 Product Migration Assistance team via https://wso2.com/contact/ before you kick start the migration. They will help you to gauge the complexity of the migration and assist you when needed.

WSO2 is committed to support their customers and community by providing the required migration support including step-by-step guides, scripts, tools, etc. and please get in touch with us via — https://wso2.com/contact/