Preface#
Due to historical reasons, the Xiaomi Mall product site has many problems and inconveniences. With the rapid technological changes and the construction of the technology department's "centralization", it is becoming less and less suitable for the rapidly iterating business needs. Next, I will explain the pain points we have encountered and the solutions to these pain points from a technical perspective, which is the story of the birth of the Phoenix framework.
Why do we need to design a framework? In fact, it is the result of business development orientation. If we don't design, we will encounter the following problems:
- Under the new product demand planning, we are unable to undertake large projects and can only make small modifications.
- Initially, each end of the Xiaomi website product site had its own set of code logic, with different styles.
- With historical accumulation, a single interface function has over 2000 lines of code, and the cost of familiarizing oneself with the code logic is increasing.
- Poor isolation, the availability of services is heavily dependent on downstream, and any fluctuations in downstream interfaces can cause panic for our interfaces.
- With the overall centralization of technology, as the number of interface calls increases, the interfaces become slower.
- The code is not decoupled, especially for new colleagues, the risk of new projects going live is high.
- Lack of maintenance for basic Go components, unable to monitor downstream interfaces in real time.
Reflection#
If we plan to refactor from a technical perspective, the first problem we need to solve is how to abstract the existing scheduling logic into a framework model that takes into account stability, convenient development, maintainability, and monitoring.
I conducted research on some open-source concurrency frameworks and found that traditional concurrency scheduling models generally have dependencies, timeout control, thread pool allocation scheduling, circuit breaking and flow control, interface monitoring, and other functions.
Why didn't we directly use open-source concurrency frameworks for development?
I found that the LiteFlow framework is the most popular and well-received framework in the industry, so I went to Github to learn about the details of the framework's underlying implementation. As I delved into reading the source code, I found that this framework is really excellent in design, but it is also too large and complex, especially the EL rule syntax, which has a certain learning curve.
So I started thinking, as a business developer, I don't want to deal with such complex dependencies. I only need to care about the interfaces of the central platform that my product site calls and their dependent interfaces. Especially for large interfaces that bundle dozens of downstream interfaces, it is almost impossible to understand the design details of each interface. If the dependency relationship is particularly complex, then the EL rules will be very complex and the maintenance cost will be extremely high.
So how do we design a lightweight, fast, efficient concurrency framework that fundamentally solves the problem of developers manually maintaining dependency relationships?
Since there are dependencies, can we automatically build dependency relationships through algorithms?
Design#
Based on the actual scenario of the product site, we found that there are several calls to downstream interfaces, and the request interfaces have different protocols and middleware.
More importantly, there are dependencies between interfaces. We found that the interface calls have a directed dependency graph structure. Therefore, we can traverse the dependency relationships and arrange concurrent groups.
This solves the problem of dependencies. We can execute the tasks of each concurrent group in parallel, so we can obtain the results of all interfaces or dependencies.
After obtaining the results, how do we arrange the business logic and isolate the downstream interfaces? In fact, the principle is very simple. Since tasks can be layered, our business calls, business orchestration, and anti-corruption layer can also be designed in layers.
- The Transfer layer is the business logic layer, used for business orchestration, providing BO data for client use.
- The Task layer is the core design layer for concurrent execution. Here, each sub-task in the concurrent group is arranged and executed, used for timeout control, time consumption statistics, and other operations.
- The Infrastructure layer is designed as an anti-corruption layer to isolate the calls to downstream interfaces. This design improves the stability of the interfaces.
Conclusion#
Well, the above text is an explanation of the "automatically building a concurrent call graph business framework", which is the Phoenix Framework.
Phoenix was initially mentioned on Zhou Zhiming's website "Phoenix Architecture". On the one hand, it is a tribute to Zhou Zhiming's architectural design understanding and learning of Java-related knowledge. On the other hand, Phoenix, like the phoenix bird, is also the life cycle of software. It is born with the rapid development of business and dies with the contraction of business, constantly regenerating.
Finally, I will explain the problems encountered by this framework and the solutions in a series. Thank you for reading. If you are interested, I recommend you to follow our official account. Let's become stronger together~
