On December 4, 2021, the 16th Academic Annual Meeting of the Shanghai Computer Society—along with the 2nd Yangtze River Delta 5G*AI Empowerment Agriculture and Tourism Industry Forum, was grandly held at the Holiday Hotel in Chongming District, Shanghai. This annual meeting was hosted by the Shanghai Computer Society and brought together many experts and scholars from the domestic computer science field, including those from Jiangyin ZhuriLimited Companywas invited to attend as a supporting unit.

At 10 a.m., Jaingyin Zhuritec, in collaboration with the Shanghai Computer Society and the Information Services Committee, jointly hosted a sub-forum on "Industrial Ecological Collaborative Operation Platform and Governance Technology". The forum specially invited Xue Guangtao, Vice Chairman and Secretary-General of the Shanghai Computer Society, Professor Cao Jian from Shanghai Jiaotong University, Professor Wang Jie from Stanford University, Professor Zeng Guosun from Tongji University, and Ma Quanyi, a community operation expert from Huawei. Through cross-industry expert reports and roundtable discussions, they exchanged on the core technologies and construction experience of the ecological system collaborative operation platform, and made a prospect for the application of big data technology in ecosystem governance.

Professor Zeng Guosun from Tongji University in Shanghai presented a report on "Open Ecological Cloud ERP Platform".

Ma Quanyi, the Director of Operations for the Huawei openEULER community, presented a report on "The Operation and Management of Open Source Communities"

Xiao Xuesong, the Technical Director of Jaingyin Zhuritec, presented a report on "Internet Solutions for the Textile and Apparel Industry Based on Open Communities".

After the breakout session, Xue Guangtao, Vice Chairman and Secretary General of the Shanghai Computer Society and Vice Dean of the School of Electronics and Information Engineering at Shanghai Jiao Tong University; Wan Fang, a Ph.D. from Donghua University in Shanghai, and Chen En, the Director of Visual Arts at Zhuritec Group, were interviewed by the media. They expressed their views on the empowerment of computer technology in the fashion industry.

At 1:30 p.m., the main forum was held. Zhu Wenyuan, the Chairman of the Shanghai Computer Society, pointed out in his opening speech: "In recent years, artificial intelligence technology, based on computer science, has developed rapidly, and some industries that are deeply integrated with artificial intelligence have achieved leapfrog development. As an important measure of the national strategy for rural revitalization, the development of agritourism has attracted widespread attention and expectations."

Zhu Wenyuan, the Chairman of the Shanghai Computer Society, delivered the opening speech

Xue Guangtao, the Vice Chairman and Secretary-General of the Shanghai Computer Society and Vice Dean of the School of Electronics and Information Engineering at Shanghai Jiao Tong University, presided over the annual conference

Jaingyin Zhuritec's Chairman Wang Wei was invited to attend

Professor Cao Jian, Distinguished Professor at Shanghai Jiao Tong University and Director of the Institute of Network and Service Computing, Department of Computer Science, delivered a speech

Zhou Guangxin, the Project Operations Director of Jaingyin Zhuritec, presented an industry report on the "Fashion Industry - IoT Integration Platform"

The Zhuritec team is a learning-oriented and agile team comprised of industry experts, academic scholars, software engineers, product architects, and operational engineering technicians. The core members of the team are interdisciplinary and multidisciplinary, focusing on practical applications and dedicated to providing services and delivering results in the fields of 5G, AI, cloud computing, big data, and the metaverse, among other computer science disciplines.

Leveraging its parent company's 20 years of brand operation and service experience in the fashion industry both domestically and internationally, along with the outcomes of collaborative efforts with the "Industry-Academic-Research Joint Laboratory" at Shanghai Jiao Tong University, Donghua University in Shanghai, and Jiangnan University, Zhuritec has acquired nearly 50 invention patents and software product copyright certificates.

Jaingyin Zhuritec adheres to a customer-centric principle, with its products serving as the linchpin, aiming to address the pain points in the fashion industry such as small batch sizes, a wide variety of products, short cycles, numerous stages, and fragmented demands. In the context of personalized consumer demand and supply, Zhuritec offers SaaS and PaaS services for scenarios including product planning, market research, event management, budget management, operation of both online and offline channels, production operations, inventory operations, and user operations. Additionally, it develops smart hardware such as PDAs and all-in-one machines in the field of IoT.

At this forum, Zhuritec also showcased to the attending experts its digital operation center and a platform that integrates technology, fashion, and industrial communities. The presentation received unanimous acclaim from the experts. Zhuritec will continue to dedicate itself to the construction of a digital ecosystem and the operation of community platforms, with the goal of promoting industrial upgrading and development.

From the extended holiday breaks at the beginning of the year leading to delayed resumption of work, to various companies implementing work-from-home arrangements, it seems everyone's life has been affected by this epidemic. Enterprises around the globe are facing a significant challenge. Impacted by the epidemic, the fashion industry has taken a considerable hit, with most brands already starting to seek new avenues, choosing to embrace "digitalization" in response.

 

1、The prolonged "Ice Age"

Impacted by the epidemic, the fashion industry has faced a considerable blow. In order to minimize losses, there's a constant stream of news about major brands closing stores, and offline sales have significantly decreased. A portion of the brands has also promptly adjusted their revenue expectations. The sudden onslaught of the epidemic has led numerous domestic and international clothing companies to enter a "Ice Age."

2、Embrace "digitalization"

After a period of darkness, most brands have begun to seek new avenues and have chosen to embrace "digitalization".

In the post-pandemic era, as consumers pursue a quality lifestyle, they not only have higher expectations for the clothing products themselves but also require further enhancement of the shopping experience throughout the entire consumption process.

In the era of informatization, management is everywhere. The use of various systems such as ERP (Enterprise Resource Planning) and OA (Office Automation) for informatized management is essentially aimed at managing the supply chain, administration, orders, etc., including managing people, goods, materials, and finances. The implementation of digital systems has changed the traditional management model.

3、Enterprise informatization construction

Enterprise informatization refers to the process by which a company utilizes modern information technology to digitize various aspects of its operations, including website construction, online marketing, production processes, product sales, employee office collaboration, financial management, staff management, customer management, and the analysis of changes in the external market environment. Through computer networks and information systems, it processes and generates new information resources for employees. This is aimed at seizing market opportunities, rationally allocating corporate resources, optimizing the combination of production factors, and thereby enhancing the economic benefits and competitiveness of the enterprise.

Jaingyin Zhuritec, a subsidiary of the Zhuri Group, is focused on the apparel and textile industry. The company is engaged in high-end customization, designer brands, lifestyle and community, operations, design, research and development, manufacturing, as well as apparel and textile manufacturing services based on informatized cross-border integration. Its main businesses include clothing import and export trade, high-end customization for men and women, cooperation with designer brands, e-commerce, online shopping malls, etc. The group is committed to delivering technology and management, integrating supply chain construction, establishing "fashion customization" and "personalized high-end customization" business models, and building the Internet+ designer brand platform, aiming to "achieve the integration of product platformization and the productization of services", making it an industry-leading enterprise that integrates apparel trade, high-end fashion design, e-commerce, management consulting, and IT product services.

Focusing on the core values of "dedication, attention, and service," Jiangyin Zhuritec Company designs its own products, innovates in technology and management, implements enterprise information construction, integrates the industrial chain, exports technology and management expertise, and promotes industrial structural transformation. It utilizes big data and machine learning to drive intelligent forecasting and process services, as well as provides visualization data services for big data and related industrial investments and operations maintenance.

The current main products and services are as follows: Zhuri Enterprise Resource Planning (ERP) System, Zhuri Enterprise Office Automation (OA) System, Zhuri Planning Service System, Zhuri Inventory Management System, Zhuri Material Service System, Tian Xia Bu Cang fabtank APP, intelligent process management system based on process engines, intelligent clothing retrieval technology based on computer vision (image search, silhouette similarity calculation), and B/S-based clothing pattern/model card/component management system.

Yibu Mini Program, independently developed by Zhuritec, is an application-based mini program that integrates the identities of buyer, store, and brand owner. Relying on the GeekZone independent designer industrialization platform, Yibu Mini Program provides a convenient tool dedicated to serving the fashion industry, offering trend information to brands and opening up brand channels. It offers independent designers a comprehensive package of project management functions, including planning, scheduling, design, progress, and other assistant services. Additionally, it provides customized enterprise-level solutions for brand owners, such as exhibitions, ordering, conferences, forums, etc., allowing fashion industry professionals to be liberated from daily trivialities and focus more on creative work.

In the intricate international environment, global industrial and supply chains are facing significant challenges. On a comprehensive level, technological innovation is reshaping the composition of factors, organizational methods, production methods, and demand structures of the entire economic ecosystem.

In recent years, Zhuri Group has been committed to building an industry community platform and leveraging the microservices philosophy of DevOps to streamline data chains, solve issues in collaborative design and rapid development, flexible manufacturing, and JIT intelligent logistics as its innovative businesses. Through the hard work and dedication of several generations of IT professionals, the Group has, at the turn of the year, successfully delivered a logistics product driven by a process rule engine-powered project activity data engine. This product integrates and presents distributed data in a centralized manner through the secondary processing of the rule engine, ultimately generating valuable report engine data.

On February 9, 2021, the "Zhuritec 2020 Product Presentation" was held in the exhibition hall on the 8th floor of Zhuri Group. Led by Bian Chengwei, the Deputy General Manager of Zhuritec IT, the entire presentation was conducted with detailed explanations. The heads of various departments within the group were invited to observe and learn from the presentation, and the venue was filled with people, creating a solemn and lively atmosphere.

With the close cooperation of the Zhuritec IT team members, a comprehensive demonstration of the industry tool product service integration and informatization products was presented to the audience. The presentation showcased the connectivity of various processes, including the operational display of logistics project products, achieving centralized management of project flows, logistics, personnel, and capital flows. Additionally, a display of hardware tool products was presented, showcasing data visualization from the group level, brand level, enterprise level, to the factory level through the visitor terminal interface. Notably, the independently developed Zhuritec MES workstation task execution system, in conjunction with the workshop platform station system and task traffic light system's data integration, presented a unified display of status data generated under process-driven conditions. This left the observers in awe of the power of big data integration, and filled them with even greater expectations for the future development of the group.

Looking ahead, the Zhuritec team will orient its goals towards users and the market, leveraging the group's current development model to integrate tools and products across the industry's upstream and downstream sectors. The team aims to set the industry benchmark and contribute to enhancing the quality and upgrading the industry overall！

The "Independent Designer Ecological Industry Strategic Alliance" jointly initiated by Zhuritec GeekZone x Li Wei was officially launched on October 13th at the Shangri-La Hotel in Shanghai. With the theme of "Technology Drives Fashion, Design Creates the Future", the event primarily introduced the gradual development of GeekZone in the fashion industry and its future planning for the designer industry, marking the official formation of the designer alliance.

 

Guests signing in for the launch event

As the event kicked off, Designer Li Wei enumerated and elaborated on the obstacles encountered by independent designers in the process of brand operation. From her own perspective, there are certain shortcomings in areas such as marketing resources, supply chain management, operating costs, business operations, and data planning. Addressing the serious challenges they have been facing, the sustainable development of brands has become an increasingly discussed topic.

Important guests at the launch event (from left to right)

Sue Ann - QI COLLECTIVE Partners

薛广涛 - Deputy Dean of the School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University

武学凯 - China's Top Ten Fashion Designers and Winners of the "Golden Top Award" for Chinese Fashion Design

曾光明 - CEO of Huayi Tomorrow Technology, Former Chief Editor of Netease Media, and Former Partner of Kuaishou Technology

李薇 - he Founder and Designer of AWAYLEE

吴宏坤 - General Manager of Shanghai Shangxin Brand Management Co., Ltd., Entrepreneurship Instructor at Beijing Institute of Fashion Technology, and External Instructor at Donghua University

王伟 - The General Manager of Zhuritec 

闫军 - KARL LAGERFELD's Chief Executive Officer and President for Greater China

陈婉 - General Manager of Women's Fashion Department, JD.com

夏明 - Master Supervisor from the School of Fashion and Textiles, Donghua University 

Designer of AWAYLEE 李薇

In September this year, designer Li Wei officially signed a contract to become the spokesperson for Zhuritec GeekZone and embarked on a series of in-depth collaborations. In response to the above issues, relevant departments of the Zhuri Group promptly formulated a comprehensive plan to support the designer's series of processes. Li Wei mentioned that through the collaboration with Zhuri Group's GeekZone during this fashion week, they were able to produce over 150 SKUs of ready-to-wear clothing in just three days, which was something she never imagined before. She expressed confidence and anticipation for the future development of design and the brand.

The spokesperson of Zhuritec Mr.Wei

Over the years, Zhuri Group has been committed to the research and development of an integrated service framework for the fashion industry, the design and development of industry-specific application software, and the construction of a mini-program app market based on an industrialization platform for independent designer brands. Adhering to craftsmanship, combining industry experience and scenarios, and utilizing big data, artificial intelligence, cloud computing, and virtual reality technology, Zhuri Group has built infrastructure based on the industry's ecosystem, providing data mining and process optimization services.

Product Director of the Yibu Mini-Program 应万贵

This launch event also marks the debut of the Yibu Mini-Program, an independently developed application by Zhuritec. The Yibu Mini-Program integrates the roles of buyer, shop, and brand manager into one single app. Relying on the GeekZone independent designer industrialization platform, the Yibu Mini-Program provides a convenient tool dedicated to serving the fashion industry, offering trend information and expanding brand channels for brands. It packages a series of project management functions for independent designers, providing comprehensive assistant services including planning, scheduling, design, and progress tracking. For brand managers, it offers customized enterprise-level solutions for exhibitions, ordering, meetings, and forums, allowing fashion industry professionals to be liberated from daily chores and return to more creative work with greater focus.

Project Director of Yibu Mini-Program 周广鑫

Through the independent designer eco-industry strategic alliance platform, Zhuritec also plans to launch the "Who's Your Companion" public welfare activity on the Yibu Mini-Program. Here, "companion" refers to both artistic companionship and spiritual support, as well as strategic partners, design resource partners, industry resource partners, channel partners, and others. We aim to gather strength with our partners to jointly promote artistic public welfare. With the mission of "integrating design art with fashion industry innovation to promote artistic public welfare," we strive to bring art and public welfare closer together, building a harmonious environment of spiritual companionship.

Public Welfare Poster

GeekCrowd Designer 陈恩

At the press conference, professors from two leading academic institutions shared their plans and prospects for collaborative development, focusing on how to better integrate current campus research and development engineering as well as technological intelligence with the fashion industry. Zhuri has been maintaining long-term friendly cooperation with Shanghai Jiao Tong University and Donghua University, providing more opportunities for outstanding graduates and creating maximum value for the technology and fashion industries.

The Deputy Dean of the School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University  薛广涛

Master Supervisor of the Fashion Institute, Donghua University 夏明

The mid-forum focused on several topics, including "How technology enterprises provide value to designer alliances," "The expression of demand for technological products from independent designer eco-industry alliances," "Viewing the integration of technology and industry from a communication perspective," and "The necessity of establishing designer alliances from the perspective of mature designer brand markets abroad."

The discussion on these relevant topics was led by four industry professionals, who each presented their own insights and expressed confidence in the independent designer eco-industry alliance. They looked forward to a future where more and more people realize the important relationship between technology and fashion, and give greater attention to the independent design market.

                                       

                                                             -The Designer Alliance Officially Launches-

Looking forward, the independent designer eco-industry strategic alliance is anticipated to foster the participation of more outstanding designers, working together to create a new and exciting future for fashion technology.

 

On the afternoon of September 24, 2016, the signing ceremony for the "Joint Laboratory of Shanghai Jiao Tong University and Zhuritec Intelligent Commercial Technology Platform" between the School of Electronic and Information Engineering of Shanghai Jiao Tong University and Jaingyin Zhuritec Company was held in the China Network Audio-visual Industry Base in Minhang District, Shanghai. This signing marks a historic step forward in the cooperation between the two parties in the field of intelligent commercial platform technology development.

At the signing ceremony, Executive Vice President Ni Jianrong of Jaingyin Zhuritec, Dean Tai Nengling, and Secretary Yang Yifan from the School of Electronic and Information Engineering of Shanghai Jiao Tong University delivered speeches respectively. Both parties expressed their strong desire to establish close cooperation. Wang Wei, the General Manager of Jaingyin Zhuritec, and Dean Xue Guangtao from the School of Electronic and Information Engineering of Shanghai Jiao Tong University signed the agreement on behalf of their respective organizations. The signing ceremony concluded successfully in a harmonious and amicable atmosphere.

It is understood that Zhuri Group, the parent company of Zhuri Information Technology Company, has gradually developed from a small foreign trade company founded in 2003 into a group enterprise with five core subsidiaries, achieving an annual sales revenue of US$60 million. Through independent design, technological innovation, management innovation, enterprise information construction, integration of the industrial chain, and the export of technology and management, the company has promoted industrial structural transformation and become an industry leader integrating garment trade, high-end fashion design, e-commerce, management consulting, and IT product services. Shanghai Jiao Tong University, as a nationally renowned university with a global reputation, possesses rich scientific research talent resources and model experience in the industrialization of scientific and technological achievements in frontier sciences. Both parties expressed full confidence and expectation for this cooperation.

After the ceremony, five major media outlets including Sina, Sohu, NetEase, and China Fashion Network conducted interviews on the cooperation. Here is the content of the interview transcripts:

1. Media: As a clothing enterprise, could you briefly introduce the characteristics of Zhuri Group?

Zhuri: Zhuri Group is jointly established by Jiangyin Jiaxiang Trading Co., Ltd. and Jiangyin Zhuritec Co., Ltd. and other four subsidiaries. Our main businesses include clothing import and export trade, high-end customization of men's and women's clothing, cooperation with designer brands, e-commerce, and online shopping malls. The group has 600 employees, production workshops, warehousing and logistics centers, multiple modern production lines, and multiple cooperative design studios. We have overseas offices in Sydney, Melbourne, New York, Los Angeles, etc. The annual output value exceeds 60 million US dollars. The group is committed to exporting technology and management, integrating the construction of supply chain integration, establishing a business model of "fashion customization" and "personalized high-end customization", creating an Internet + designer brand platform, and aiming at the integration of "realizing the platformization of products and the productization of services". Jiangyin Zhuritec Co., Ltd. has achieved rapid development in the service fields of management consulting, information services, enterprise collaboration, and industrial chain resource integration. We have developed multiple software products such as enterprise-level application information platforms, customized user terminals, and smart engineering, striving to become an industry model for Made in China 2025.

2. Media: After listening to the host's introduction of your company, Zhuri Group, as a comprehensive group company in the clothing industry, how did you come to have the intention to cooperate with the Software School of Shanghai Jiao Tong University?

Zhuri: Our group is a company that has grown from the grassroots, having experienced the financial crisis in 2008 and the major reshuffle in the clothing industry. In 2010, the company began its information construction by continuously attracting talents in the industry, integrating the industrial chain and transforming its structure, and embarking on the path of independent design, technological innovation, and management innovation. Especially in recent years, with the rapid development of the internet, the company deeply understands the importance of informationization, platformization, and craftsmanship to the innovation and future development of its business model. We believe that companies need the internet, especially strong data support behind it, which is conducive to the company's precise positioning of products and target customers. This has prompted us to seek higher education institutions with solid technical foundations, strong academic atmospheres, strong professionalism, and high focus on research and development to meet the company's future strategic planning. The rigorous, down-to-earth, and innovative school spirit of Shanghai Jiao Tong University, as well as its talent advantages in gathering science and engineering elites and its extensive experience in developing domestic and international business projects, have deeply attracted us. After our initial contact and communication, we hit it off immediately.

3. Media: What aspects of the domestic clothing industry platform does the joint laboratory established between Shanghai Jiao Tong University and Zhuri Group focus on?

Zhuri's Response: Our current collaboration primarily revolves around a research, education, and industry project focused on intelligent applications in the clothing sector. The project itself will leverage the strengths of Shanghai Jiao Tong University in teaching, research, and talent cultivation, while incorporating the development needs of our company, to propose the construction of a business intelligence platform based on the internet environment. Specifically, on one hand, we will cooperate in areas such as big data analysis and recommendation system applications, as well as intelligent workflow systems. On the other hand, we will strive to make breakthroughs in translating campus research achievements into market demands, seeking more points of collaboration, and ultimately forming a market-oriented approach that closely integrates research, education, and industry technology transformation. We aim to establish a long-term partnership that leads to mutual benefits for both parties and beyond.

4. Media question: Just now, I heard from the host that Zhuri Group currently has commercial sections such as Geek Fashion, Geek Online, and Geek Community in China. Can you introduce the differences between these Geek commercial contents and traditional clothing enterprises?

Zhuri's answer: The term geek originated from foreign computer hackers and now refers to a group of people full of imagination and creativity, characterized by the pursuit of personality realization and self creation. Our group aims to attract more people to become geeks by gathering this group of people who pursue the ultimate life experience, conveying the concept of geek lifestyle, creating new and diverse personalized lifestyles. Starting from the clothing industry, create a geek fashion circle that can be personalized, self actualized, and based on sensory experience. Compared with traditional clothing enterprises, Geek Fashion is more flexible and versatile, with the characteristics of "strong personalized symbols and full experience", which can trigger various consumption phenomena such as "loser's comeback", "wild lilies also have spring", "everyone is a designer", and "Cinderella also has crystal shoes".

5. Media question: What kind of clothing group does Zhuri Group ultimately want to create through its cooperation with Shanghai Jiao Tong University and strong alliance? What is the impact on other domestic clothing enterprises?

Zhuri's answer: "Setting lofty goals and cherishing the world" is the development philosophy that our group has always adhered to. The group has gained recognition from customers and partners in the international fast fashion consumer market through years of hard work. At the beginning of this year, we adjusted our strategic plan and began the large-scale development of cross-border e-commerce in sync with the domestic market. Our group collaborates with Jiaotong University to develop an intelligent business platform with the ultimate goal of forming a comprehensive information processing platform guided by market demand, with fast response and dynamic resource allocation. As for the impact on other domestic clothing enterprises? It can only be said that we have accelerated the application of Industry 4.0 in the domestic clothing industry, and we look forward to more domestic clothing enterprises joining us in the path of transformation, innovation, and development.

In the complex international environment, global industrial and supply chains are facing significant impacts; Overall, technological innovation is reshaping the composition of elements, organizational methods, production methods, and demand structure of the entire economic ecosystem.

In recent years, Zhuri Group has been innovating its business by creating an industry community platform, using DevOps microservice concept, connecting data chains, solving collaborative design and rapid development, flexible manufacturing, and JIT intelligent logistics. Through the efforts of several generations of IT personnel, at the end of the year and beginning of the year, it has successfully completed logistics products driven by process rule engines for project activity data engines. Discrete distributed data is processed by rule engines for secondary processing, and report engine data is presented in a centralized manner.

On February 9, 2021, at the exhibition hall on the 8th floor of Zhuri Group, the ZhuritecIT team conducted a "Zhuritec2020 Product Report Exhibition". Zhuritec Vice President Bian Chengwei gave a full explanation, and the heads of various business units of the group were invited to observe and learn on site. The atmosphere was grand and enthusiastic.

With the close cooperation of all members of the ZhuritecIT team, the on-site personnel were showcased with industry tools, product services, integrated information technology products, and the connection methods of each link were used to showcase one by one: the operation display of logistics project products, achieving centralized management of project flow, logistics, personnel flow, and capital flow; And the display of hardware tool products, displaying data at the group level, brand level, enterprise level, and factory level through visitor machine screens. Among them, Zhuritec's independently developed large MES workstation task execution system, combined with the data integration of the workshop platform workstation system and task traffic light system, uniformly displays the state data generated under process driving, allowing on-site observers to marvel at the integration of big data and have high expectations for the development of the group.

In the future, the Zhuritec team will target users, target the market, and combine the current development model of the group to connect the upstream and downstream product tool clusters in the industry, striving to become an industry benchmark and help improve the quality and upgrading of the industry!

The evolution of high concurrency distributed architecture on the server side

1.Summary

This article takes Taobao as an example to introduce the evolution process of server architecture from 100 concurrent to 10 million level concurrent scenarios, and lists the relevant technologies encountered in each evolution stage to give everyone a comprehensive understanding of architecture evolution. Finally, the article summarizes some principles of architecture design.

 

2.Basic concepts

Before introducing architecture, in order to avoid some readers being unfamiliar with some concepts in architecture design, the following are some of the most basic concepts:

• 分布式
  系统中的多个模块在不同服务器上部署，即可称为分布式系统，如Tomcat和数据库分别部署在不同的服务器上，或两个相同功能的Tomcat分别部署在不同服务器上

• 高可用
  系统中部分节点失效时，其他节点能够接替它继续提供服务，则可认为系统具有高可用性

• 集群
  一个特定领域的软件部署在多台服务器上并作为一个整体提供一类服务，这个整体称为集群。如Zookeeper中的Master和Slave分别部署在多台服务器上，共同组成一个整体提供集中配置服务。在常见的集群中，客户端往往能够连接任意一个节点获得服务，并且当集群中一个节点掉线时，其他节点往往能够自动的接替它继续提供服务，这时候说明集群具有高可用性

• 负载均衡
  请求发送到系统时，通过某些方式把请求均匀分发到多个节点上，使系统中每个节点能够均匀的处理请求负载，则可认为系统是负载均衡的

• 正向代理和反向代理
  系统内部要访问外部网络时，统一通过一个代理服务器把请求转发出去，在外部网络看来就是代理服务器发起的访问，此时代理服务器实现的是正向代理；当外部请求进入系统时，代理服务器把该请求转发到系统中的某台服务器上，对外部请求来说，与之交互的只有代理服务器，此时代理服务器实现的是反向代理。简单来说，正向代理是代理服务器代替系统内部来访问外部网络的过程，反向代理是外部请求访问系统时通过代理服务器转发到内部服务器的过程。

   

3.架构演进

3.1 单机架构

以淘宝作为例子。在网站最初时，应用数量与用户数都较少，可以把Tomcat和数据库部署在同一台服务器上。浏览器往www.taobao.com发起请求时，首先经过DNS服务器（域名系统）把域名转换为实际IP地址10.102.4.1，浏览器转而访问该IP对应的Tomcat。

随着用户数的增长，Tomcat和数据库之间竞争资源，单机性能不足以支撑业务

3.2 第一次演进：Tomcat与数据库分开部署

Tomcat和数据库分别独占服务器资源，显著提高两者各自性能。

随着用户数的增长，并发读写数据库成为瓶颈

3.3 第二次演进：引入本地缓存和分布式缓存

在Tomcat同服务器上或同JVM中增加本地缓存，并在外部增加分布式缓存，缓存热门商品信息或热门商品的html页面等。通过缓存能把绝大多数请求在读写数据库前拦截掉，大大降低数据库压力。其中涉及的技术包括：使用memcached作为本地缓存，使用Redis作为分布式缓存，还会涉及缓存一致性、缓存穿透/击穿、缓存雪崩、热点数据集中失效等问题。

缓存抗住了大部分的访问请求，随着用户数的增长，并发压力主要落在单机的Tomcat上，响应逐渐变慢

3.4 第三次演进：引入反向代理实现负载均衡

在多台服务器上分别部署Tomcat，使用反向代理软件（Nginx）把请求均匀分发到每个Tomcat中。此处假设Tomcat最多支持100个并发，Nginx最多支持50000个并发，那么理论上Nginx把请求分发到500个Tomcat上，就能抗住50000个并发。其中涉及的技术包括：Nginx、HAProxy，两者都是工作在网络第七层的反向代理软件，主要支持http协议，还会涉及session共享、文件上传下载的问题。

反向代理使应用服务器可支持的并发量大大增加，但并发量的增长也意味着更多请求穿透到数据库，单机的数据库最终成为瓶颈

3.5 第四次演进：数据库读写分离

把数据库划分为读库和写库，读库可以有多个，通过同步机制把写库的数据同步到读库，对于需要查询最新写入数据场景，可通过在缓存中多写一份，通过缓存获得最新数据。其中涉及的技术包括：Mycat，它是数据库中间件，可通过它来组织数据库的分离读写和分库分表，客户端通过它来访问下层数据库，还会涉及数据同步，数据一致性的问题。

业务逐渐变多，不同业务之间的访问量差距较大，不同业务直接竞争数据库，相互影响性能

3.6 第五次演进：数据库按业务分库

把不同业务的数据保存到不同的数据库中，使业务之间的资源竞争降低，对于访问量大的业务，可以部署更多的服务器来支撑。这样同时导致跨业务的表无法直接做关联分析，需要通过其他途径来解决，但这不是本文讨论的重点，有兴趣的可以自行搜索解决方案。

随着用户数的增长，单机的写库会逐渐会达到性能瓶颈

3.7 第六次演进：把大表拆分为小表

比如针对评论数据，可按照商品ID进行hash，路由到对应的表中存储；针对支付记录，可按照小时创建表，每个小时表继续拆分为小表，使用用户ID或记录编号来路由数据。只要实时操作的表数据量足够小，请求能够足够均匀的分发到多台服务器上的小表，那数据库就能通过水平扩展的方式来提高性能。其中前面提到的Mycat也支持在大表拆分为小表情况下的访问控制。

这种做法显著的增加了数据库运维的难度，对DBA的要求较高。数据库设计到这种结构时，已经可以称为分布式数据库，但是这只是一个逻辑的数据库整体，数据库里不同的组成部分是由不同的组件单独来实现的，如分库分表的管理和请求分发，由Mycat实现，SQL的解析由单机的数据库实现，读写分离可能由网关和消息队列来实现，查询结果的汇总可能由数据库接口层来实现等等，这种架构其实是MPP（大规模并行处理）架构的一类实现。

目前开源和商用都已经有不少MPP数据库，开源中比较流行的有Greenplum、TiDB、Postgresql XC、HAWQ等，商用的如南大通用的GBase、睿帆科技的雪球DB、华为的LibrA等等，不同的MPP数据库的侧重点也不一样，如TiDB更侧重于分布式OLTP场景，Greenplum更侧重于分布式OLAP场景，这些MPP数据库基本都提供了类似Postgresql、Oracle、MySQL那样的SQL标准支持能力，能把一个查询解析为分布式的执行计划分发到每台机器上并行执行，最终由数据库本身汇总数据进行返回，也提供了诸如权限管理、分库分表、事务、数据副本等能力，并且大多能够支持100个节点以上的集群，大大降低了数据库运维的成本，并且使数据库也能够实现水平扩展。

数据库和Tomcat都能够水平扩展，可支撑的并发大幅提高，随着用户数的增长，最终单机的Nginx会成为瓶颈

3.8 第七次演进：使用LVS或F5来使多个Nginx负载均衡

由于瓶颈在Nginx，因此无法通过两层的Nginx来实现多个Nginx的负载均衡。图中的LVS和F5是工作在网络第四层的负载均衡解决方案，其中LVS是软件，运行在操作系统内核态，可对TCP请求或更高层级的网络协议进行转发，因此支持的协议更丰富，并且性能也远高于Nginx，可假设单机的LVS可支持几十万个并发的请求转发；F5是一种负载均衡硬件，与LVS提供的能力类似，性能比LVS更高，但价格昂贵。由于LVS是单机版的软件，若LVS所在服务器宕机则会导致整个后端系统都无法访问，因此需要有备用节点。可使用keepalived软件模拟出虚拟IP，然后把虚拟IP绑定到多台LVS服务器上，浏览器访问虚拟IP时，会被路由器重定向到真实的LVS服务器，当主LVS服务器宕机时，keepalived软件会自动更新路由器中的路由表，把虚拟IP重定向到另外一台正常的LVS服务器，从而达到LVS服务器高可用的效果。

此处需要注意的是，上图中从Nginx层到Tomcat层这样画并不代表全部Nginx都转发请求到全部的Tomcat，在实际使用时，可能会是几个Nginx下面接一部分的Tomcat，这些Nginx之间通过keepalived实现高可用，其他的Nginx接另外的Tomcat，这样可接入的Tomcat数量就能成倍的增加。

由于LVS也是单机的，随着并发数增长到几十万时，LVS服务器最终会达到瓶颈，此时用户数达到千万甚至上亿级别，用户分布在不同的地区，与服务器机房距离不同，导致了访问的延迟会明显不同

3.9 第八次演进：通过DNS轮询实现机房间的负载均衡

在DNS服务器中可配置一个域名对应多个IP地址，每个IP地址对应到不同的机房里的虚拟IP。当用户访问www.taobao.com时，DNS服务器会使用轮询策略或其他策略，来选择某个IP供用户访问。此方式能实现机房间的负载均衡，至此，系统可做到机房级别的水平扩展，千万级到亿级的并发量都可通过增加机房来解决，系统入口处的请求并发量不再是问题。

随着数据的丰富程度和业务的发展，检索、分析等需求越来越丰富，单单依靠数据库无法解决如此丰富的需求

3.10 第九次演进：引入NoSQL数据库和搜索引擎等技术

当数据库中的数据多到一定规模时，数据库就不适用于复杂的查询了，往往只能满足普通查询的场景。对于统计报表场景，在数据量大时不一定能跑出结果，而且在跑复杂查询时会导致其他查询变慢，对于全文检索、可变数据结构等场景，数据库天生不适用。因此需要针对特定的场景，引入合适的解决方案。如对于海量文件存储，可通过分布式文件系统HDFS解决，对于key value类型的数据，可通过HBase和Redis等方案解决，对于全文检索场景，可通过搜索引擎如ElasticSearch解决，对于多维分析场景，可通过Kylin或Druid等方案解决。

当然，引入更多组件同时会提高系统的复杂度，不同的组件保存的数据需要同步，需要考虑一致性的问题，需要有更多的运维手段来管理这些组件等。

引入更多组件解决了丰富的需求，业务维度能够极大扩充，随之而来的是一个应用中包含了太多的业务代码，业务的升级迭代变得困难

3.11 第十次演进：大应用拆分为小应用

按照业务板块来划分应用代码，使单个应用的职责更清晰，相互之间可以做到独立升级迭代。这时候应用之间可能会涉及到一些公共配置，可以通过分布式配置中心Zookeeper来解决。

不同应用之间存在共用的模块，由应用单独管理会导致相同代码存在多份，导致公共功能升级时全部应用代码都要跟着升级

3.12 第十一次演进：复用的功能抽离成微服务

如用户管理、订单、支付、鉴权等功能在多个应用中都存在，那么可以把这些功能的代码单独抽取出来形成一个单独的服务来管理，这样的服务就是所谓的微服务，应用和服务之间通过HTTP、TCP或RPC请求等多种方式来访问公共服务，每个单独的服务都可以由单独的团队来管理。此外，可以通过Dubbo、SpringCloud等框架实现服务治理、限流、熔断、降级等功能，提高服务的稳定性和可用性。

不同服务的接口访问方式不同，应用代码需要适配多种访问方式才能使用服务，此外，应用访问服务，服务之间也可能相互访问，调用链将会变得非常复杂，逻辑变得混乱

3.13 第十二次演进：引入企业服务总线ESB屏蔽服务接口的访问差异

通过ESB统一进行访问协议转换，应用统一通过ESB来访问后端服务，服务与服务之间也通过ESB来相互调用，以此降低系统的耦合程度。这种单个应用拆分为多个应用，公共服务单独抽取出来来管理，并使用企业消息总线来解除服务之间耦合问题的架构，就是所谓的SOA（面向服务）架构，这种架构与微服务架构容易混淆，因为表现形式十分相似。个人理解，微服务架构更多是指把系统里的公共服务抽取出来单独运维管理的思想，而SOA架构则是指一种拆分服务并使服务接口访问变得统一的架构思想，SOA架构中包含了微服务的思想。

业务不断发展，应用和服务都会不断变多，应用和服务的部署变得复杂，同一台服务器上部署多个服务还要解决运行环境冲突的问题，此外，对于如大促这类需要动态扩缩容的场景，需要水平扩展服务的性能，就需要在新增的服务上准备运行环境，部署服务等，运维将变得十分困难

3.14 第十三次演进：引入容器化技术实现运行环境隔离与动态服务管理

目前最流行的容器化技术是Docker，最流行的容器管理服务是Kubernetes(K8S)，应用/服务可以打包为Docker镜像，通过K8S来动态分发和部署镜像。Docker镜像可理解为一个能运行你的应用/服务的最小的操作系统，里面放着应用/服务的运行代码，运行环境根据实际的需要设置好。把整个“操作系统”打包为一个镜像后，就可以分发到需要部署相关服务的机器上，直接启动Docker镜像就可以把服务起起来，使服务的部署和运维变得简单。

在大促的之前，可以在现有的机器集群上划分出服务器来启动Docker镜像，增强服务的性能，大促过后就可以关闭镜像，对机器上的其他服务不造成影响（在3.14节之前，服务运行在新增机器上需要修改系统配置来适配服务，这会导致机器上其他服务需要的运行环境被破坏）。

使用容器化技术后服务动态扩缩容问题得以解决，但是机器还是需要公司自身来管理，在非大促的时候，还是需要闲置着大量的机器资源来应对大促，机器自身成本和运维成本都极高，资源利用率低

3.15 第十四次演进：以云平台承载系统

系统可部署到公有云上，利用公有云的海量机器资源，解决动态硬件资源的问题，在大促的时间段里，在云平台中临时申请更多的资源，结合Docker和K8S来快速部署服务，在大促结束后释放资源，真正做到按需付费，资源利用率大大提高，同时大大降低了运维成本。

所谓的云平台，就是把海量机器资源，通过统一的资源管理，抽象为一个资源整体，在之上可按需动态申请硬件资源（如CPU、内存、网络等），并且之上提供通用的操作系统，提供常用的技术组件（如Hadoop技术栈，MPP数据库等）供用户使用，甚至提供开发好的应用，用户不需要关系应用内部使用了什么技术，就能够解决需求（如音视频转码服务、邮件服务、个人博客等）。在云平台中会涉及如下几个概念：

• IaaS：基础设施即服务。对应于上面所说的机器资源统一为资源整体，可动态申请硬件资源的层面；

• PaaS：平台即服务。对应于上面所说的提供常用的技术组件方便系统的开发和维护；

• SaaS：软件即服务。对应于上面所说的提供开发好的应用或服务，按功能或性能要求付费。

至此，以上所提到的从高并发访问问题，到服务的架构和系统实施的层面都有了各自的解决方案，但同时也应该意识到，在上面的介绍中，其实是有意忽略了诸如跨机房数据同步、分布式事务实现等等的实际问题，这些问题以后有机会再拿出来单独讨论

4.架构设计总结

 

• 架构的调整是否必须按照上述演变路径进行？
  不是的，以上所说的架构演变顺序只是针对某个侧面进行单独的改进，在实际场景中，可能同一时间会有几个问题需要解决，或者可能先达到瓶颈的是另外的方面，这时候就应该按照实际问题实际解决。如在政府类的并发量可能不大，但业务可能很丰富的场景，高并发就不是重点解决的问题，此时优先需要的可能会是丰富需求的解决方案。

• 对于将要实施的系统，架构应该设计到什么程度？
  对于单次实施并且性能指标明确的系统，架构设计到能够支持系统的性能指标要求就足够了，但要留有扩展架构的接口以便不备之需。对于不断发展的系统，如电商平台，应设计到能满足下一阶段用户量和性能指标要求的程度，并根据业务的增长不断的迭代升级架构，以支持更高的并发和更丰富的业务。

• 服务端架构和大数据架构有什么区别？
  所谓的“大数据”其实是海量数据采集清洗转换、数据存储、数据分析、数据服务等场景解决方案的一个统称，在每一个场景都包含了多种可选的技术，如数据采集有Flume、Sqoop、Kettle等，数据存储有分布式文件系统HDFS、FastDFS，NoSQL数据库HBase、MongoDB等，数据分析有Spark技术栈、机器学习算法等。总的来说大数据架构就是根据业务的需求，整合各种大数据组件组合而成的架构，一般会提供分布式存储、分布式计算、多维分析、数据仓库、机器学习算法等能力。而服务端架构更多指的是应用组织层面的架构，底层能力往往是由大数据架构来提供。

• 有没有一些架构设计的原则？

• N+1设计。系统中的每个组件都应做到没有单点故障；

• 回滚设计。确保系统可以向前兼容，在系统升级时应能有办法回滚版本；

• 禁用设计。应该提供控制具体功能是否可用的配置，在系统出现故障时能够快速下线功能；

• 监控设计。在设计阶段就要考虑监控的手段；

• 多活数据中心设计。若系统需要极高的高可用，应考虑在多地实施数据中心进行多活，至少在一个机房断电的情况下系统依然可用；

• 采用成熟的技术。刚开发的或开源的技术往往存在很多隐藏的bug，出了问题没有商业支持可能会是一个灾难；

• 资源隔离设计。应避免单一业务占用全部资源；

• 架构应能水平扩展。系统只有做到能水平扩展，才能有效避免瓶颈问题；

• 非核心则购买。非核心功能若需要占用大量的研发资源才能解决，则考虑购买成熟的产品；

• 使用商用硬件。商用硬件能有效降低硬件故障的机率；

• 快速迭代。系统应该快速开发小功能模块，尽快上线进行验证，早日发现问题大大降低系统交付的风险；

• 无状态设计。服务接口应该做成无状态的，当前接口的访问不依赖于接口上次访问的状态。

分布式事务的解决方案有如下几种：

• 全局消息

• 基于可靠消息服务的分布式事务

• TCC

• 最大努力通知

方案1：全局事务（DTP模型）

   全局事务基于DTP模型实现。DTP是由X/Open组织提出的一种分布式事务模型——X/Open Distributed Transaction Processing Reference Model。它规定了要实现分布式事务，需要三种角色：

• AP：Application 应用系统

  它就是我们开发的业务系统，在我们开发的过程中，可以使用资源管理器提供的事务接口来实现分布式事务。

• TM：Transaction Manager 事务管理器

• 分布式事务的实现由事务管理器来完成，它会提供分布式事务的操作接口供我们的业务系统调用。这些接口称为TX接口。

• 事务管理器还管理着所有的资源管理器，通过它们提供的XA接口来同一调度这些资源管理器，以实现分布式事务。

• DTP只是一套实现分布式事务的规范，并没有定义具体如何实现分布式事务，TM可以采用2PC、3PC、Paxos等协议实现分布式事务。

• RM：Resource Manager 资源管理器

• 能够提供数据服务的对象都可以是资源管理器，比如：数据库、消息中间件、缓存等。大部分场景下，数据库即为分布式事务中的资源管理器。

• 资源管理器能够提供单数据库的事务能力，它们通过XA接口，将本数据库的提交、回滚等能力提供给事务管理器调用，以帮助事务管理器实现分布式的事务管理。

• XA是DTP模型定义的接口，用于向事务管理器提供该资源管理器(该数据库)的提交、回滚等能力。

• DTP只是一套实现分布式事务的规范，RM具体的实现是由数据库厂商来完成的。

1. 有没有基于DTP模型的分布式事务中间件？

2. DTP模型有啥优缺点？

方案2：基于可靠消息服务的分布式事务

   这种实现分布式事务的方式需要通过消息中间件来实现。假设有A和B两个系统，分别可以处理任务A和任务B。此时系统A中存在一个业务流程，需要将任务A和任务B在同一个事务中处理。下面来介绍基于消息中间件来实现这种分布式事务。

• 在系统A处理任务A前，首先向消息中间件发送一条消息

• 消息中间件收到后将该条消息持久化，但并不投递。此时下游系统B仍然不知道该条消息的存在。

• 消息中间件持久化成功后，便向系统A返回一个确认应答；

• 系统A收到确认应答后，则可以开始处理任务A；

• 任务A处理完成后，向消息中间件发送Commit请求。该请求发送完成后，对系统A而言，该事务的处理过程就结束了，此时它可以处理别的任务了。
  但commit消息可能会在传输途中丢失，从而消息中间件并不会向系统B投递这条消息，从而系统就会出现不一致性。这个问题由消息中间件的事务回查机制完成，下文会介绍。

• 消息中间件收到Commit指令后，便向系统B投递该消息，从而触发任务B的执行；

• 当任务B执行完成后，系统B向消息中间件返回一个确认应答，告诉消息中间件该消息已经成功消费，此时，这个分布式事务完成。

上述过程可以得出如下几个结论：

1. 消息中间件扮演者分布式事务协调者的角色。

2. 系统A完成任务A后，到任务B执行完成之间，会存在一定的时间差。在这个时间差内，整个系统处于数据不一致的状态，但这短暂的不一致性是可以接受的，因为经过短暂的时间后，系统又可以保持数据一致性，满足BASE理论。

上述过程中，如果任务A处理失败，那么需要进入回滚流程，如下图所示：

• 若系统A在处理任务A时失败，那么就会向消息中间件发送Rollback请求。和发送Commit请求一样，系统A发完之后便可以认为回滚已经完成，它便可以去做其他的事情。

• 消息中间件收到回滚请求后，直接将该消息丢弃，而不投递给系统B，从而不会触发系统B的任务B。

此时系统又处于一致性状态，因为任务A和任务B都没有执行。

   上面所介绍的Commit和Rollback都属于理想情况，但在实际系统中，Commit和Rollback指令都有可能在传输途中丢失。那么当出现这种情况的时候，消息中间件是如何保证数据一致性呢？——答案就是超时询问机制。

   系统A除了实现正常的业务流程外，还需提供一个事务询问的接口，供消息中间件调用。当消息中间件收到一条事务型消息后便开始计时，如果到了超时时间也没收到系统A发来的Commit或Rollback指令的话，就会主动调用系统A提供的事务询问接口询问该系统目前的状态。该接口会返回三种结果：

• 提交

  若获得的状态是“提交”，则将该消息投递给系统B。

• 回滚

  若获得的状态是“回滚”，则直接将条消息丢弃。

• 处理中

  若获得的状态是“处理中”，则继续等待。

消息中间件的超时询问机制能够防止上游系统因在传输过程中丢失Commit/Rollback指令而导致的系统不一致情况，而且能降低上游系统的阻塞时间，上游系统只要发出Commit/Rollback指令后便可以处理其他任务，无需等待确认应答。而Commit/Rollback指令丢失的情况通过超时询问机制来弥补，这样大大降低上游系统的阻塞时间，提升系统的并发度。

   下面来说一说消息投递过程的可靠性保证。
当上游系统执行完任务并向消息中间件提交了Commit指令后，便可以处理其他任务了，此时它可以认为事务已经完成，接下来消息中间件一定会保证消息被下游系统成功消费掉！那么这是怎么做到的呢？这由消息中间件的投递流程来保证。

   消息中间件向下游系统投递完消息后便进入阻塞等待状态，下游系统便立即进行任务的处理，任务处理完成后便向消息中间件返回应答。消息中间件收到确认应答后便认为该事务处理完毕！

   如果消息在投递过程中丢失，或消息的确认应答在返回途中丢失，那么消息中间件在等待确认应答超时之后就会重新投递，直到下游消费者返回消费成功响应为止。当然，一般消息中间件可以设置消息重试的次数和时间间隔，比如：当第一次投递失败后，每隔五分钟重试一次，一共重试3次。如果重试3次之后仍然投递失败，那么这条消息就需要人工干预。

有的同学可能要问：消息投递失败后为什么不回滚消息，而是不断尝试重新投递？

   这就涉及到整套分布式事务系统的实现成本问题。
我们知道，当系统A将向消息中间件发送Commit指令后，它便去做别的事情了。如果此时消息投递失败，需要回滚的话，就需要让系统A事先提供回滚接口，这无疑增加了额外的开发成本，业务系统的复杂度也将提高。对于一个业务系统的设计目标是，在保证性能的前提下，最大限度地降低系统复杂度，从而能够降低系统的运维成本。

不知大家是否发现，上游系统A向消息中间件提交Commit/Rollback消息采用的是异步方式，也就是当上游系统提交完消息后便可以去做别的事情，接下来提交、回滚就完全交给消息中间件来完成，并且完全信任消息中间件，认为它一定能正确地完成事务的提交或回滚。然而，消息中间件向下游系统投递消息的过程是同步的。也就是消息中间件将消息投递给下游系统后，它会阻塞等待，等下游系统成功处理完任务返回确认应答后才取消阻塞等待。为什么这两者在设计上是不一致的呢？

   首先，上游系统和消息中间件之间采用异步通信是为了提高系统并发度。业务系统直接和用户打交道，用户体验尤为重要，因此这种异步通信方式能够极大程度地降低用户等待时间。此外，异步通信相对于同步通信而言，没有了长时间的阻塞等待，因此系统的并发性也大大增加。但异步通信可能会引起Commit/Rollback指令丢失的问题，这就由消息中间件的超时询问机制来弥补。

   那么，消息中间件和下游系统之间为什么要采用同步通信呢？

   异步能提升系统性能，但随之会增加系统复杂度；而同步虽然降低系统并发度，但实现成本较低。因此，在对并发度要求不是很高的情况下，或者服务器资源较为充裕的情况下，我们可以选择同步来降低系统的复杂度。
我们知道，消息中间件是一个独立于业务系统的第三方中间件，它不和任何业务系统产生直接的耦合，它也不和用户产生直接的关联，它一般部署在独立的服务器集群上，具有良好的可扩展性，所以不必太过于担心它的性能，如果处理速度无法满足我们的要求，可以增加机器来解决。而且，即使消息中间件处理速度有一定的延迟那也是可以接受的，因为前面所介绍的BASE理论就告诉我们了，我们追求的是最终一致性，而非实时一致性，因此消息中间件产生的时延导致事务短暂的不一致是可以接受的。

方案3：最大努力通知（定期校对）

   最大努力通知也被称为定期校对，其实在方案二中已经包含，这里再单独介绍，主要是为了知识体系的完整性。这种方案也需要消息中间件的参与，其过程如下：

• 上游系统在完成任务后，向消息中间件同步地发送一条消息，确保消息中间件成功持久化这条消息，然后上游系统可以去做别的事情了；

• 消息中间件收到消息后负责将该消息同步投递给相应的下游系统，并触发下游系统的任务执行；

• 当下游系统处理成功后，向消息中间件反馈确认应答，消息中间件便可以将该条消息删除，从而该事务完成。

上面是一个理想化的过程，但在实际场景中，往往会出现如下几种意外情况：

1. 消息中间件向下游系统投递消息失败

2. 上游系统向消息中间件发送消息失败

   对于第一种情况，消息中间件具有重试机制，我们可以在消息中间件中设置消息的重试次数和重试时间间隔，对于网络不稳定导致的消息投递失败的情况，往往重试几次后消息便可以成功投递，如果超过了重试的上限仍然投递失败，那么消息中间件不再投递该消息，而是记录在失败消息表中，消息中间件需要提供失败消息的查询接口，下游系统会定期查询失败消息，并将其消费，这就是所谓的“定期校对”。

   如果重复投递和定期校对都不能解决问题，往往是因为下游系统出现了严重的错误，此时就需要人工干预。

   对于第二种情况，需要在上游系统中建立消息重发机制。可以在上游系统建立一张本地消息表，并将 任务处理过程 和 向本地消息表中插入消息 这两个步骤放在一个本地事务中完成。如果向本地消息表插入消息失败，那么就会触发回滚，之前的任务处理结果就会被取消。如果这量步都执行成功，那么该本地事务就完成了。接下来会有一个专门的消息发送者不断地发送本地消息表中的消息，如果发送失败它会返回重试。当然，也要给消息发送者设置重试的上限，一般而言，达到重试上限仍然发送失败，那就意味着消息中间件出现严重的问题，此时也只有人工干预才能解决问题。

   对于不支持事务型消息的消息中间件，如果要实现分布式事务的话，就可以采用这种方式。它能够通过重试机制+定期校对实现分布式事务，但相比于第二种方案，它达到数据一致性的周期较长，而且还需要在上游系统中实现消息重试发布机制，以确保消息成功发布给消息中间件，这无疑增加了业务系统的开发成本，使得业务系统不够纯粹，并且这些额外的业务逻辑无疑会占用业务系统的硬件资源，从而影响性能。

   因此，尽量选择支持事务型消息的消息中间件来实现分布式事务，如RocketMQ。

方案4：TCC（两阶段型、补偿型）

TCC即为Try Confirm Cancel，它属于补偿型分布式事务。顾名思义，TCC实现分布式事务一共有三个步骤：

• Try：尝试待执行的业务

• 这个过程并未执行业务，只是完成所有业务的一致性检查，并预留好执行所需的全部资源

• Confirm：执行业务

• 这个过程真正开始执行业务，由于Try阶段已经完成了一致性检查，因此本过程直接执行，而不做任何检查。并且在执行的过程中，会使用到Try阶段预留的业务资源。

• Cancel：取消执行的业务

• 若业务执行失败，则进入Cancel阶段，它会释放所有占用的业务资源，并回滚Confirm阶段执行的操作。

下面以一个转账的例子来解释下TCC实现分布式事务的过程。

假设用户A用他的账户余额给用户B发一个100元的红包，并且余额系统和红包系统是两个独立的系统。

• Try

• 创建一条转账流水，并将流水的状态设为交易中

• 将用户A的账户中扣除100元（预留业务资源）

• Try成功之后，便进入Confirm阶段

• Try过程发生任何异常，均进入Cancel阶段

• Confirm

• 向B用户的红包账户中增加100元

• 将流水的状态设为交易已完成

• Confirm过程发生任何异常，均进入Cancel阶段

• Confirm过程执行成功，则该事务结束

• Cancel

• 将用户A的账户增加100元

• 将流水的状态设为交易失败

   在传统事务机制中，业务逻辑的执行和事务的处理，是在不同的阶段由不同的部件来完成的：业务逻辑部分访问资源实现数据存储，其处理是由业务系统负责；事务处理部分通过协调资源管理器以实现事务管理，其处理由事务管理器来负责。二者没有太多交互的地方，所以，传统事务管理器的事务处理逻辑，仅需要着眼于事务完成（commit/rollback）阶段，而不必关注业务执行阶段。

TCC全局事务必须基于RM本地事务来实现全局事务

   TCC服务是由Try/Confirm/Cancel业务构成的，其Try/Confirm/Cancel业务在执行时，会访问资源管理器（Resource Manager，下文简称RM）来存取数据。这些存取操作，必须要参与RM本地事务，以使其更改的数据要么都commit，要么都rollback。

这一点不难理解，考虑一下如下场景：

   假设图中的服务B没有基于RM本地事务（以RDBS为例，可通过设置auto-commit为true来模拟），那么一旦[B:Try]操作中途执行失败，TCC事务框架后续决定回滚全局事务时，该[B:Cancel]则需要判断[B:Try]中哪些操作已经写到DB、哪些操作还没有写到DB：假设[B:Try]业务有5个写库操作，[B:Cancel]业务则需要逐个判断这5个操作是否生效，并将生效的操作执行反向操作。

   不幸的是，由于[B:Cancel]业务也有n（0<=n<=5）个反向的写库操作，此时一旦[B:Cancel]也中途出错，则后续的[B:Cancel]执行任务更加繁重。因为，相比第一次[B:Cancel]操作，后续的[B:Cancel]操作还需要判断先前的[B:Cancel]操作的n（0<=n<=5）个写库中哪几个已经执行、哪几个还没有执行，这就涉及到了幂等性问题。而对幂等性的保障，又很可能还需要涉及额外的写库操作，该写库操作又会因为没有RM本地事务的支持而存在类似问题。。。可想而知，如果不基于RM本地事务，TCC事务框架是无法有效的管理TCC全局事务的。

   反之，基于RM本地事务的TCC事务，这种情况则会很容易处理：[B:Try]操作中途执行失败，TCC事务框架将其参与RM本地事务直接rollback即可。后续TCC事务框架决定回滚全局事务时，在知道“[B:Try]操作涉及的RM本地事务已经rollback”的情况下，根本无需执行[B:Cancel]操作。

   换句话说，基于RM本地事务实现TCC事务框架时，一个TCC型服务的cancel业务要么执行，要么不执行，不需要考虑部分执行的情况。

TCC事务框架应该提供Confirm/Cancel服务的幂等性保障

   一般认为，服务的幂等性，是指针对同一个服务的多次(n>1)请求和对它的单次(n=1)请求，二者具有相同的副作用。

   在TCC事务模型中，Confirm/Cancel业务可能会被重复调用，其原因很多。比如，全局事务在提交/回滚时会调用各TCC服务的Confirm/Cancel业务逻辑。执行这些Confirm/Cancel业务时，可能会出现如网络中断的故障而使得全局事务不能完成。因此，故障恢复机制后续仍然会重新提交/回滚这些未完成的全局事务，这样就会再次调用参与该全局事务的各TCC服务的Confirm/Cancel业务逻辑。

   既然Confirm/Cancel业务可能会被多次调用，就需要保障其幂等性。
   那么，应该由TCC事务框架来提供幂等性保障？还是应该由业务系统自行来保障幂等性呢？
   个人认为，应该是由TCC事务框架来提供幂等性保障。如果仅仅只是极个别服务存在这个问题的话，那么由业务系统来负责也是可以的；然而，这是一类公共问题，毫无疑问，所有TCC服务的Confirm/Cancel业务存在幂等性问题。TCC服务的公共问题应该由TCC事务框架来解决；而且，考虑一下由业务系统来负责幂等性需要考虑的问题，就会发现，这无疑增大了业务系统的复杂度。

智能过程自动化共包含五项关键技术，本文将阐明如何用其来提高生产力及效率、降低运营风险并改善客户体验。

自 2007 年 9 月金融危机以来，为了同时提高成本利用率、客户满意度及员工敬业度，很多公司应用了精益管理，并且有很多项目在各个方面取得了实质性影响。但数字化的进展却变得更失衡。

例如在保险行业中，2016 年 10 月的一份 FIS 研究发现，在受调查的保险公司中，有 99.6％的保险公司承认在数字创新实现中面临阻碍，而 80％的公司认为他们需要数字能力来应对业务挑战。而在 2016 年，「insurtech」投资的繁荣使这个难题变得更加复杂——自 2015 年以来，111 个交易中的融资额超过了 35 亿美元。

随着宏观经济条件对各行业利润率的继续施压，提高成本生产力和释放新价值重返高级管理议程首位。而问题在于，我们还能做什么？

这就是智能过程自动化（IPA）出现的原因。我们相信它将是公司下一代运营模式的核心部分，许多跨行业的公司都在尝试 IPA，并取得了显著成果：

50％至 70％的任务实现自动化…………转化为 20％至 35％的年运行成本效率…………且直通处理时长减少 50％至 60％…………投资回报率的百分比通常为三位数。

新的技术承诺同年回报能达到两位甚至三位数，对于这一点我们应保持怀疑。但经验表明，如果执行人员能够仔细思考并理解机会的驱动因素，且能将它与其它驱动下一代运营模式的方法和能力有效结合，那么 IPA 的承诺就是真实的（了解更多相关信息，请参阅「数字领域的下一代运营模式」）。

什么是智能过程自动化？

IPA 的本质是「从人类手中夺走机器人」，其核心则是一套将基本过程重设与机器人过程自动化及机器学习相结合的新技术。它是一套业务流程的改进，也是下一代通过删除重复、可复制以及常规任务来帮助知识型工作者的工具；它还能简化交互与加快进程来从根本上改善客户体验。

IPA 可模仿人类活动，并能随时间推移而优化学习。由于深度学习和认知技术的进步，基于规则性自动化的传统杠杆的决策能力得以增强。IPA 承诺从根本上提高效率、提升工人绩效、减少操作风险，以及改善响应时间与客户使用体验。

IPA 共包括五项核心技术：

机器人过程自动化（RPA）：一种软件自动化工具，它能自动执行常规任务，如通过现存用户界面进行数据提取与清理。机器人有一个与人类相同的用户 ID，并能执行基于规则的任务，如访问电子邮件和系统、执行计算、创建文档和报告，以及检查文件。RPA 帮一家大型保险合作社削减了每天影响着 2500 个高风险账户的多余排队程序，并释放出 81％的 FTE 来取代主动型帐户管理职位。智能工作流程：一种管理过程的软件工具，它能集成人类和机器组执行的任务（如在 RPA 之上帮助管理过程）。这使用户得以实时启动和跟踪端到端过程的状态；该软件将会管理不同组别的切换，包括机器人和人类用户之间的切换，并提供瓶颈之处的统计数据。机器学习/高级分析：一种通过「监督」和「无监督」学习来识别结构化数据中的模式（如日常性能数据）的算法。监督算法在开始根据自己的新输入做出预测之前，会从输入和输出的结构化数据集来学习；而无监督算法会观察结构化数据，并开始提供对已识别模式的洞见。机器学习和高级分析可能会改变保险公司的游戏规则，如在提高合规性、降低成本结构及从新的洞见中获得优势的竞争中进行改变。高级分析已在领先的人力资源部门中广泛实施，来确定及评估领导和管理人员的核心品质，以便更好地预测行为，发展职业道路及规划领导权继任。自然语言生成（NLG）：一种通过遵循将观察结果从数据转化为散文的规则以在人类与技术之间创建无缝交互的软件引擎。广播公司一直在使用自然语言生成来实时起草游戏情节，而结构化的性能数据被传输至自然语言引擎中，来自动编写内部和外部的管理报告。一家大型金融机构已在使用 NLG 来复写每周的管理报告。认知代理：这项技术将机器学习和自然语言生成相结合，来构建一个完全虚拟的劳动力（或称「代理」）；这个代理能够执行任务、沟通、从数据集中学习，甚至可以根据「情绪检测」做出决策。认知代理可以通过电话或聊天来帮助员工和客户，如应用在员工服务中心。使用认知技术的英国汽车保险公司的转化率提高了 22％，验证错误率降低了 40％，整体投资回报率为 330％。

IPA 运行时的情况会怎样？下面举一个保险公司的例子：在那里人们命令处理器从 13 个不同系统中提取数据，保证服务「一切正常」。

机器人可以通过使用 IPA 来代替手动点击（RPA）、解释大量文本的通信（NLG）、制定规则下不必预编程的决策（机器学习）、提供客户建议（认知代理），并提供对系统和人员之间相互切换的实时跟踪（智能工作流程）。

IPA 的价值

当 IPA 接管了机械重复的任务之后，工作人员可专注于提高客户满意度，思考如何让来自企业外部——如新闻、事件、社交媒体、嵌入式传感器以及类似的任何地方——的有价值数据帮助实现业务目标。

尽管实现完整的 IPA 套件能够带来全方位的裨益，但是公司可以通过实现 IPA 中的个别技术就能快速释放显著的价值。如图表 1 所示，仅 RPA 就可以带动生产力显著 增长。

图表 1

一家大型金融机构采用规模化的 RPA 转型，将记录到报告流程中的 60％至 70％的任务自动化，并使年度运营效率提高了 30% 甚至更高。使用相同的方法，另一个机构在超额排队过程中降低了 80％的处理成本。FT500 中的另一家金融机构使用机器人技术每年降低 1.75 亿英镑的成本，减招 120 多个全职员工 (FTE)。

另外，IPA 还可以帮助领导者从在众多复杂系统中几十年的投资中获得最大利益，并且同时做出许多复杂的决策。我们也看到企业插入控制以实时激活由其他的新发现触发的其他过程。例如，创建一个耦合了自然语言生成引擎的非监督机器学习平台，可以很快地允许处理结构化地日常性能数据以生成能够帮助领导者做决策的真知灼见，同时改变内部管理过程。不再需要难以进行的功能有限的报告流程，仅仅将它们累放在桌面上就行。尤其在保险行业中，IPA 可以发挥巨大作用。

如何开始你的 IPA 改革呢？

IPA 不需要大量的基础设施投资，因为它处理的是企业信息系统的表示层。例如，存在于信息系统顶层的 RPA 软件，可以在不改变信息技术后端的情况下实现快速返回。某些情况下，企业可以在两个星期内建造并运行 RPA 系统并且从中获益。

根据我们的经验，以下步骤在推动成功的大规模 IPA 改革中是最重要的：

1. 迅速协调 IPA 在运营模型中的作用

任何一个有效的 IPA 倡议必须基于明确理解企业的总体战略和它的下一代运转模型在帮助实现 IPA 的过程中所起的作用。这需要清楚地阐明目标结束状态和过程，以专注于实现它。这种清晰度允许商业领袖评估和协调实现运转模型的方法和能力。在许多情况下，IPA 在推动变革方面发挥着重要的甚至主导作用，但是当公司理解了 IPA 和公司的其他能力和方法能够共同发挥作用的时候，IPA 便能发挥其最大价值。自动化即将到来，现在是定义可能的艺术并将其战略性地应用在最有意义的地方的时候。

2. 围绕整个 IPA 解决方案组合设计，以最大限度地发挥作用

企业不应该仅仅浅尝辄止一部分 IPA 技术。世界前进太快以至于没有有效利用 IPA，其包含的所有技术协同工作时才能产生全面影响。

企业需要设想和实施整体的优化规划以使得投资回报最大化。尽管在信息孤岛上实施自动化项目是比较容易和快速的，但是这种方法存有缺陷。单独的技术不足以依靠自身去获取价值。事实上，为了转变一个集团的工作方式，需要重设基本的过程。

应制定实施的详细路线图，以确定所有自动化增强的机会，并允许企业通过平衡它们对规模化扩展初始用况的解决方案的影响来有序执行 IPA 举措。通过快速创建当前任务的概述以及执行这些任务所需的资源和能力，开始你的 IPA 之旅。然后部署经验丰富的环网孵化器团队，根据对业务线和 IPA 功能的深入理解重新设计过程和团队流程。

3. 快速开发一款最小可行产品 (MVP)

尽管设计一个全面的 IPA 组合是重要的，但是刚开始的时候一次解决所有问题难免令人生畏。许多管理人员被许多复杂的数据仓库项目所淹没，其中一些项目需要十年才能完成，并且大大超出预算。与其他数字化工作一样，最好选择

一个偏好速度和影响的端到端的过程或者用户流程，以用 IPA 来重新设计并加强业务过程，然后努力推出最小的可行产品 (MVP)，它是产品的最简化版本，但依然能完成任务。这样，你就可以快速地测试什么是有效和无效的，并作出针对性改变。

IPA 可以在几周而不是几年内以更少的错误和和更少的员工的「忙碌工作」的形式提供切实的价值。早期试点项目的快速回报有助于获得利益相关方和执行发起人的支持，以便采用更深入的计划以通过全面 IPA 改革来利用可实现的潜力。

4. 建立动力，捕获价值

任何 IPA 的实现都应该结合快速获胜和更大的长期发展。具体的细节路线应该根植于对生产自动化模型顺序进行的过程的重新设计，以及对集团营利的方式的重新设想。

例如，保险行业中的每一个产品线都有不同程度可以被标准化和自动化的潜力，并且需要核查以及排序 (如图表 2 所示)。看一些交易、保险业与定价、政策管理、债权融资和会计中的时间密集型流程，当决定了它们如何在未来发挥作用时，就从一个空白表单开始。

图表 2

5. 嵌入持久的能力，实现可持续发展

持续创造价值的一个成功途径就是创建一个卓越中心（CoE）来管理 IPA 转型，并通过能力建设、认证、标准、供应商管理以及创建可重用解决方案模式库来支持 IPA 解决方案的快速部署。这样的 CoE 应当位于中心，并且尺寸可以相当小，因为它可以调用现有的精益管理或过程优化 CoE，而业务所有权和执行应当位于业务线或数字工厂。

系统控制需要到位，企业应该在业务线中嵌入关键业务分析和数字技能，以便他们能够拥有该流程。他们还需要重新设计组织结构以捕获价值，建立未来状态的运作模型以扩大其 IPA 举措，为未来结构制定蓝图，获取影响并嵌入新能力，以及提供培训和讲习班来解释为什么手动过程的自动化将解放团队以专注于更多的创造性活动。

在此过程中吸引你的企业和职能团队至关重要——例如，通过建立机器人——并建立可重用的资产，如手册。建立持久的 IPA 能力的最成功的方法是结合教练、在职培训和知识共享。为了在企业级获得价值，企业需要在 IPA 水平、流程重新设计、精益原则以及领域专业知识方面具有深厚技能的人。仅有技能本身是不够的。许多组织选择引入外部支持来补充他们的人才库并加速企业转型。

6. 谨慎地协调转型的管理和沟通

与任何大型转型计划一样，需要一个强大的沟通计划来帮助管理重新部署、激发兴奋、并协调变革故事与企业战略。建立新的执行模型方面的成功将取决于它与组织文化以及人们能够适应敏捷实践的程度的多少相关。此外，变革冠军需要在内部发掘，以使变革成功。

公司正在以极低的成本利用 IPA 投资和开发新平台、吸引客户以及争取顾问，但这些多是无关紧要的东西。将来的胜利者是那些拥抱这些新能力，将其作为下一代运行模型，并快速从中获利的人，而不是那些放不开手的落伍者。