There are many pitfalls in legacy application modernization if migration strategy is wrong.
Incorrect strategy/method may lead to large amount of bugs, insufficient test coverage, low maintainability, degradation in performance, etc.

    

Problems with Manual Migration

Most organizations rewrite their system functions 1 by 1 manually, with very high cost and very low quality.
This no longer makes sense as we now have AI.

    

Problems with Straight Conversion

Straight conversion does not modernize in the right way

・Line-by-line replacing legacy language means new system processes in the same way as legacy, legacy disadvantages remain.

Special Java system not only leads to low maintainability, slows down modification, but also becomes a major obstacle to using generative AI, hindering future improvement.

Quality challenges

・Customer provides test data, not knowing if data is enough or not for conversion test.
・Difficult comprehensive test is conducted by customer or 3rd party, without focusing on specifics of the conversion.
・Difficult performance problems to deal with.

Customer burden

・Conversion without understanding the logic behind.
   ˃ some code cannot be converted correctly (bug creation), or even fails to be converted;
   ˃ customers deal with those issues manually;

Straight conversion poses big risk on quality, cost, and delivery schedule.

    

Problems with Large Language Model (LLM)

Even though LLM is a popular general-purpose generative AI model, it has its own issues to be resolved. Most reputable modernization vendors do not use LLM.

LLM Limitation

1, It requires extensive learning.
　・Extremely large amount of high-quality training data required v.s. insufficient amount of source code in COBOL and other legacy languages that are available for training purposes.
　・LLM evaluation requires extremely fast processing v.s. CPUs are too slow to meet the demand.
　　Difficult to evaluate by COBOL/Java comparison tests.
2, Additional learning for each round of bug fixes changes conversion rules.
　System test must be done over and over again.

Problem with LLM Assisted Development

Although only the parts that are not automatically converted need be developed manually, the entire source code must be checked and understood before corrections can be made, so there is no significant savings in man-hour.
Since there are many human errors in understanding the source code and coding in target language, huge amount of testing is still required.