Question: What core values can cutting-edge technologies represented by AI contribute to industrial manufacturing, especially industrial vision scenarios?
Answer: Industrial vision is the core area of industrial automation, including key tasks such as detection, identification, measurement, and positioning. AI technology that is good at solving visual perception problems has become an important entry point for creating value. Today, the global manufacturing industry is facing tremendous industrial upgrading pressure. Except for a few large-scale production companies in the head, most manufacturing production lines face the contradiction of rapid iteration of demand and insufficient information, automation, and flexibility. On the one hand, the rapidly changing global market calls for on-demand production, on-demand customization, and on-demand iteration. High-efficiency production line; on the other hand, difficult data collection, difficult connection, low degree of automation, lack of collaboration between automated stations, difficult quantitative evaluation and precise attribution of yield and other practical problems, which together constitute the overall challenge of improving manufacturing efficiency .
Most manufacturing companies need to make large strides to complement the three shortcomings of informatization, automation, and intelligence in a relatively short construction period to achieve leapfrog development. Cutting-edge technologies such as the Internet of Things, big data, machine vision, automatic planning and decision-making, and automatic control are the key technologies in this process. Among them, AI-related technologies are the basis for the leap from “manufacturing” to “intelligent manufacturing”.
Question: Specific to industrial vision scenarios, why does industrial vision need artificial intelligence?
Industrial vision must solve the problem of how to “see” the manufacturing scene and how to “perceive” and “understand” the key information. At present, optical imaging technology, multi-sensor fusion technology, and light-sensing and optical processing chip technologies have greatly improved, making “seeing smaller and clearer” a reality. On this basis, AI can just help us “perceive more accurately and understand more deeply.”
Taking the electronics manufacturing industry as an example, the annual labor cost of workstations related to visual inspection on related production lines in my country is about 6 billion yuan. The cutting-edge technology will upgrade the visual inspection process from a labor-intensive work station that is more extensive and difficult to quantify to a fully automatic work station that can be accurately quantified, fully traceable, and can intelligently integrate data, bringing production efficiency and products Significant improvement in quality.
Question: What kind of experience is it to plan, define, and design a product or solution in the field of industrial vision?
The challenge is great. Speaking of experience, “Take one’s mind and one’s muscles and bones”. In the field of industrial vision, the forward development of a product will go through demand investigation, overall plan design, key technology verification, subsystem plan design, prototype trial production, on-site plan verification, product release and market promotion. Products are required at each stage The manager leads the team in the right direction and devotes all their energy to polishing and optimization.
In the above process, the product manager’s thinking needs to be switched between the macro and the micro at any time: macro thinking is required when considering customer needs and product positioning; when considering technology implementation, it is necessary to focus on the technical details of each key level; when laying out product lines It is necessary to consider the matching of differentiated configurations to various customers; it is necessary to pay attention to specific data and customization requirements during on-site program verification.
But being a product manager in this field is also full of accomplishments: industrial vision products, especially automated vision equipment, are the most fulfilling products. When the product is successfully delivered and runs stably on the customer’s production line, the customer’s affirmation is the best return to the product development team.
Question: What are the product laws in this field that are different from other fields?
First of all, there are many demand items: in the field of industrial manufacturing, the two indicators that customers pay most attention to are capacity and quality. Around these two seemingly contradictory indicators, the product design of Industrial Vision can be disassembled into multiple design indicators:
- The main indicators related to production capacity include upstream and downstream handover methods, product/equipment/instrument TT (tack time) time, product stability (MTBF, MTBR), model switching time, etc.
- The main indicators related to quality include detection/measurement accuracy, over and missed detection, repeatability, etc.
- Due to the special environment (constant temperature, constant humidity, ultra-clean, safety) of the industrial site, the main indicators that need to be considered include product ESD, weight, emergency stop, FFU, etc.
Customers in the industry will give detailed specifications during the plan communication phase, sometimes even covering hundreds to thousands of specification definitions.
Second, there are many sources of demand: products in the industrial manufacturing field have distinct characteristics of enterprise-level (ToB) products, and the decision-making party, purchaser, and user are usually not the same team. For example, the decision-making party may be the engineering department or the equipment manufacturing department, the purchaser is the purchasing department, and the user is the production department. The customer departments at different stages are different, and different departments have different requirements for products.
Third, the degree of customization is high: Because different customers have different production line layouts, different line speeds, different upstream and downstream process equipment, and even different elevator heights (determining the highest height of the equipment), the products delivered to each customer are different. Great customization. However, the core functions of the same industrial vision product series should remain stable.
Question: Industrial vision technology can be abstracted into several aspects, such as how to “see”, how to “perceive/understand”, how to “plan/decision”, and how to “execute”. Take “Look” as an example, what level of imaging technology and lighting technology have reached today?
From the perspective of the imaging chip (more refined, better, wider, faster):
- The improvement of the chip process level has made CMOS chips with large target surfaces become mainstream industrial camera chips. The evolution of 2M→12M→29M→60M→71M→150M enables industrial inspection to reach micron-level accuracy, high full well capacity, and high dynamic range. , The feature of low noise greatly improves the imaging quality of industrial cameras;
- The high-precision coating process achieves pixel-level coating. Polarization cameras and hyperspectral cameras based on the above technologies can obtain more dimensional information of the products to be tested in industrial inspection scenarios;
- TDI technology can greatly reduce the exposure time, increase the scanning frequency of the camera, and cooperate with the high-speed image capture card to carry the visual inspection on the high-speed automated production line.
From the perspective of lighting source: Ten years ago, the light source products of the machine vision industry were basically monopolized by Japanese companies. Today, domestic machine vision light source manufacturers are rising rapidly, and the types and quality of light source products continue to improve. At present, common LED light sources (bar light, ring light, coaxial light, dome light, backlight) and light source controllers have been widely used in various vision systems, and domestic suppliers are also actively cooperating with various needs and scene lighting Verification and light source scheme. At the same time, various domestic light source suppliers actively promote the research and development of independent light sources and even vision systems, such as multi-angle line light sources, line scan time-sharing exposure systems, and integrated vision controllers. In addition, projection structured light and laser line light sources are also used in a number of 3D contour or defect detection scenarios.
Dimension of imaging schemes: There are various imaging schemes for industrial vision products and equipment. Area array schemes (stationary or flying shot detection), linear array schemes, line laser scanning, coded structured light, white light confocal and other schemes have been maturely applied. However, complex multi-station inspection equipment often integrates the above-mentioned multiple solutions to achieve higher imaging rate coverage and 3D profile measurement. It is believed that in the near future, new technologies such as computational imaging, hyperspectral imaging, and light field cameras will be further integrated into industrial vision solutions.
Question: How do “light” and “optics” affect the technical realization of specific projects? If you want to make good use of optical technology in your products, what issues should you focus on in design?
As an optical engineer, “Garbage in, garbage out” is the basic concept, and the impact of optical systems on industrial vision projects is also evident.
Overall scheme level: Complex industrial vision equipment basically integrates multiple imaging schemes, so there will also be multi-station designs. Only when the optical scheme is determined can the station distribution and equipment layout of the equipment be determined.
The optical system has constraints on the structure and automation design of the product: driven by demand, industrial vision products have high resolution, and at the same time, DFX requirements such as debug and maintainability of the equipment must be considered. The vision system usually needs to reserve an adjustment mechanism. Qualified optical engineers will design the optical plan while outputting the visual plan’s constraints on machine jitter, positioning repeatability, freedom of adjustment of the visual mechanism, adjustment range of each degree, flatness of the stage and other indicators. Only when these indicators are clear can it be possible Make downstream design not reworked and debugging easier.
The optical system has index constraints on algorithm performance: in industrial inspection scenarios, the core indicators of industrial vision inspection equipment are defect detection rate and inspection accuracy. For example, the customer requires a 90% defect detection rate, which can generally be disassembled as x% defect imaging rate multiplied by y% defect detection rate in imaging. The product of the two indicators is about 90%, and the single indicator needs to be much higher. When performing imaging verification, optical engineers need to communicate closely with algorithm engineers to confirm whether defect imaging meets the requirements of algorithm detection to ensure that the defect detection rate is the ultimate goal.
In general, we believe that the development and design of industrial vision products must be optical first. In addition to designing optical solutions, vision engineers also need to invest a lot of energy on advanced imaging methods, advanced product research and other topics. For example, in the LCD era, OLED products and processes are pre-researched, and QLED and Micro LED are pre-researched in the OLED era, so that the accumulation of knowledge will not fall behind in the process of continuous changes in upstream products and processes. Simply put, vision engineers should be the same as automation engineers—or even need to know more about processes and processes.
Question: At the “perception” level, to what extent does the industrial vision software and hardware components need to perceive the workstation scene to meet business needs?
Perception has multiple meanings: in the detection scene, the visual product realizes the accurate detection of defects, which can replace manual labor quickly and well; in the alignment scene, the vision product recognizes the object to be grasped/assembled/crimped and accurately feedbacks the object position ; In the measurement scenario, the visual product measures the geometric quantities that are the key to the process, and feeds back accurate measurement results. The above scenarios all require the reasonable selection, design and configuration of visual algorithms.
For the detection scene, the basis of perception needs to be established on the customer’s manual detection benchmark. For various products and processes to be tested, such as film materials, coil materials, and glass, as long as there is a manual testing station, there will be specific and detailed manual testing standards. It is the first step (feature detection, object detection) to detect “suspected anomalies” on the image through various algorithms/models. After that, how to understand the customer’s manual judgment logic and design the corresponding algorithm logic is more important. For example, common manual judgment standards will stipulate the length, width, area, point group distance, depth, etc. of defects, and our algorithm logic also needs to be designed based on this.
For measurement scenarios, common methods include 2D and 3D measurement. This type of scene first needs to confirm the detection accuracy required by the customer, and then disassemble key indicators such as the resolution of the vision system and the accuracy of the measurement algorithm. There were customers who used the two-dimensional (projection image measuring instrument) equipment as the reference equipment for measurement. If all the measuring equipment passed the acceptance, they needed to be compared with the measurement results of the two-dimensional equipment-in this case, even if a certain equipment The measurement result has reached the limit of its measurement principle, but if it cannot match the reference equipment, it will not be delivered smoothly.
Alignment scenarios are common in processes such as assembly, grabbing, placement, and punching. This also requires first confirming the alignment accuracy required by the customer. The positioning accuracy required by the electronics manufacturing industry has reached the micron level. The calibration, feature recognition, coordinate calculation and other links of the alignment vision component require sub-pixel accuracy, and some scenes require multiple alignments to ensure accuracy.
Question: How to choose and combine different perception technologies for a good industrial vision product or solution?
Comprehensive ability is the key. Manufacturing customers are more inclined to assign the entire production line or the visual products in the entire production line to a solution supplier/equipment manufacturer for integration. In this context, if you want to get a better order, you must specialize in one type of visual product, and at the same time have the control/design/development capabilities of other visual products or programs. For good industrial vision products or solutions, they must have complete resolution capabilities for various visual scenarios, based on traditional algorithms such as 2D/3D measurement and feature detection, and rely on deep learning-based detection, classification, and segmentation. These complex scenes are differentiated.
Question: Compared with traditional computer graphics and traditional computer vision technology, what are the advantages of the new generation of AI technology represented by deep learning? What is the relationship between deep learning technology and traditional technology in solving industrial vision problems?
For example, in some appearance defect inspection projects, only traditional algorithms are used in the early stage. Under the premise of ensuring the defect detection rate, the over-inspection rate is relatively high, and the manual re-judgment workload of the customer is relatively large, and it is not for the customer. Reduce too much manpower. After analyzing the over-inspected images, it is found that the over-inspection is mainly caused by dirt, dust, etc., and these over-inspection sources are difficult to distinguish from real defects by traditional algorithms. Through the introduction of deep learning algorithms, the over-inspection suppression effect based on deep learning classification is verified. After several rounds of model optimization (on-site over-inspection image return→model training and update model→on-site verification and continued feedback of over-inspection images), the over-inspection rate Significantly reduced, customers are very satisfied.
As another example, the appearance inspection of complete mobile phones has always been a very difficult direction in the field of electronics manufacturing. Few manufacturers dared to try before 2017. After 2017, with the diversification of vision device hardware solutions (strobes, flying shots, 6-axis robots), manufacturers gradually began to try in the market. The mobile phone has many functional modules (cameras, earpieces, speakers, buttons, charging holes), various shapes (glass, metal, mirror, frosted, chamfered, curved surfaces), and various types of defects (combined scenes of all appearance defects). The imaging situation is complex, and it is difficult to cover all defects with traditional feature detection-based algorithms. Algorithms based on deep learning have received very good feedback in trials in recent years. I believe that in the near future, there will be a mature product that can surpass the extremely high technical threshold for the appearance of the whole machine.
This shows that mature and available machine vision products with reliable performance must complement traditional algorithms and deep learning algorithms. For example, the average tact time in the panel industry can be as short as 2.5s. Under high line speed requirements, the algorithm detection time of a single product (100Mb data volume) needs to be controlled within 1.5s. At this time, traditional algorithms are usually better than deep learning. The algorithm has a speed advantage. On the other hand, for classification and complex scenes that are difficult to implement with traditional algorithms, deep learning algorithms are easier to show off.
Question: At the level of “planning/decision-making” and “execution”, how should a good industrial vision component interact with the overall automation system of industrial manufacturing? What are the more difficult product and solution design problems?
The interaction between the “planning/decision” level and the “execution” level can be intuitively understood as the interaction and handshake between the vision system and the automation system of the whole machine, or the interaction and handshake between the upper computer software and the lower computer board/PLC. From the perspective of functional division, the vision system is theoretically only responsible for vision-related actions, and PLC controls all motion axes, solenoid valves, and sensors of the machine.
Therefore, at the control level, the initialization of the camera/capturing card, the brightness of the light source, the adjustment of the brightness of the light source, the storage and detection of images are completely controlled by the vision system, the product handling/handover, the movement of each axis, and the on-off of the air circuit , The control of air pressure/temperature/safety grating/code scanning/emergency stop reset start button is completely controlled by PLC.
At the interactive level, image collection and detection result feedback require communication interaction between upper and lower computers. When the PLC receives the product ID returned by the scanning device, it will send the ID to the host computer to perform image naming and storage; when the product moves to the preset image capture position, the PLC will notify the host computer that the image can be captured and Wait for the upper computer to complete the acquisition signal, and then execute the subsequent process; after the upper computer completes the image detection, the OK/NG result of the product will be fed back to the PLC, which is convenient for product classification and blanking.
When there are a large number of visual stations in the equipment, the interaction between the upper and lower computers includes the communication between the upper computers of each visual station and the communication interaction between each visual station and the PLC. Because the operation flow of each visual station is parallel, and each The products of the workstations are not consistent (there may be multiple products in the equipment at the same time), so the interaction scheme will become very complicated.
Question: Chuangxin Qizhi has undertaken many typical industrial vision projects. In these projects, what do you think is the biggest technical challenge?
Innovative Qizhi not only provides core software and algorithms in industrial vision projects, but also integrates self-developed or purchased machines and equipment. In this process, the biggest challenge is how to cooperate tacitly between the software algorithm team based on machine vision and the automation team based on optoelectronics. For example, high-precision AOI automatic optical inspection requires a complete optical solution, a stable and anti-vibration machine, a motion mechanism with high repeat positioning accuracy, and a precise linkage between upstream and downstream to match the production line beat. Software and hardware must be coordinated to achieve the best results.
Another technical challenge is to quickly adapt to the up-to-date testing/measurement products. The industrial industry has diverse product categories and rapid product iteration. Taking the electronics manufacturing industry as an example, the switching of models requires adjusting the product carrier, adjusting the position of the vacuum nozzle of the transport mechanism, adjusting the camera working distance/lens focus/light source position, adjusting the image capture position, adjusting the probe depression position, and switching software Dozens of operations such as templates, adjustment of detection parameters, optimization of algorithm models, and adjustment of unwinding positions. Every time you switch, the machine vision model and the customized part of the overall plan must quickly adapt to the new scene.
Question: The innovative ManuVision industrial vision platform connects the processes of “perception/understanding”, “planning/decision” and “execution” into a complete technology platform, which greatly reduces the difficulty of developing and implementing industrial vision solutions. From a product perspective, what value can the ManuVision platform contribute to industrial vision scenarios?
The design concept of the ManuVision platform is to make industrial vision product development faster and to make industrial vision project delivery lighter. The ManuVision platform includes three key functional modules: Designer, Runtime and Trainer.
The Runtime module is the equipment business execution module. Through the interface of this module, you can observe the equipment capacity, each station image, detection results, abnormal information, operation logs, etc. in real time; when the equipment switches the corresponding product, the corresponding business process can be switched through the Runtime module And deep learning models.
The Designer module is a configuration module for detection schemes and business processes. The Designer module encapsulates the core operations in the industrial vision process into functional blocks. By intuitively adding and connecting functional blocks, the delivery team and the customer’s production line equipment engineers can quickly build a complete business process.
The Trainer module is preset with pre-trained models. Production line equipment engineers, QC, and operators do not need any algorithmic foundation. They only need to use the annotation tool to complete the defect annotation, and the Trainer module can automatically complete the optimization and testing of the preset model. The model can be deployed in the runtime interface with one click.
In general, the ManuVision industrial vision platform integrates our overall thinking on industrial vision technology and products into a unified software framework. It is an efficient tool for industrial vision scenes to upgrade from “manufacturing” to “intelligent manufacturing”.
