In this post, I would like to talk about devices capable of acquiring images in the Terahertz spectral range, an emergingtechnology with great potential for implementation in industry, especially in the agri-food sector.
Currrently, machine vision systems used in industry work with different ranges of the electromagnetic spectrum, such as visible light, infrared, ultraviolet, among others, which are not able to pass through matter. Therefore, these technologies can only examine the surface characterisitcs of a product or packaging, but cannot provide information from the inside.
In contrast, there are other technologies that do allow us to examine certain properties inside matter, such as metal detectors, magnetic resonance imaging, ultrasound and X-rays. Metal detectors are only capable of detecting the presence of metals. Magnetic resonance equipment is expensive and large, mainly used in medicine, and its integration at industrial level is practically unfeasible. Ultrasound equipment requires contact, requires some skill in its application and is difficult to interpret, so it is not feasible in the industrial sector. Finally, X-rays are a very dangerous ionising radiation, which implies a great effort in protective coatings and an exhaustive control of the radiation dose. Although they can pass through matter, X-rays can only provide information about the different parts of a product that absorb radiation in this range of the electromagnetic spectrum.
From this point of view, we are faced with a very important challenge, to investigate the potential of new technologies with the capacity to inspect, safely and without contact, the inside of products and packaging, obtaining relevant information on the internal characteristics, such as quality, condition, presence or absence of elements inside, homogeneity,etc.
Looking at the options, the solution may lie in promoting the integration in industry of new technologies that work in non-ionising spectral ranges with the ability to penetrate matter, such as the terahertz/near-microwave spectral range.
First radiological image in histroy. The hand of Röntgen´s wife
In 1985, Professor Röntgen took the first radiological image in history, his wife´s hand. 127 years have passed and research is still going on. In 1995, the first image in the Terhaertz range was captures, son only 27 years have passed since then. This shows the degree of maturity of Terahertz technology, still in its early stages of research. This radiation is not new, we know it is there, but today it is very difficult to generate and detect it. The main research work has focused on improving the way this radiation is emitted and captured in a coherent way, using equipment developed in the laboratory.
In recent years things have changed, new optical sensors and new terahertz sources with a very high industrialisation capcity have been obtained, which opens the doors of industry to this technology. Now there is still a very important task of research to see the scope of this technology in the different areas of industry.
CARTIF is committed to this technology and is currently working on the development of the industrial research project AGROVIS, “Intelligent VISual Computing for products/processes in the AGRI-food sector“, a project funded by the Junta de Castilla y León, framed in the field of computer vision (digital enabler of industry 4.0) associated with the agri-food sector, where one of the main objectives is to explore the different possibilities for automatically inspecting the interior of agri-food products safely.
Machine vision is one of the enablers of Industry 4.0 with increased integration in production lines, especially in the quality control of products and processes. In recent years, a real revolution is taking place in this field with the integration of Artificial Intelligence in image processing, with a potential yet to be discovered. Despite the limitations of Artificial Intelligence in terms of reliability, results are being obtained in industry that were previously unthinkable using traditional machine vision.
The purpose of this post is not to talk about the possibilities of Artificial Intelligence, as there are many blogs that deal with this task, the purpose is to highlight the potential of traditional machine vision when you have experience and develop good ideas.
Machine vision is not just a set of algorithms that are applied directly to images obtained by high-performance cameras. When we develop a machine vision system, we do so to detect a variety of defects or product characteristics. Our task is to select the most appropriate technology and generate the optimal conditions in the scene in order to extract the required information from the physical world from the captured images. There are many variables to consider in this task: the characteristics of the lighting used in the scene; the relative position between the acquisition equipment, the lighting system and the object to be analysed; the characteristics of the inspection area; the configuration and sensitivity of the acquisition systems, etc.
As a representative anecdote of the importance of experience, I would like to highlight a case that was given to us in an automative components factory.
The company had installed a high-performance commercial vision system whose objective was to identify various parts based on colour. After several failures, we were asked to help configure the equipment, but instead of acting on these devices, we worked on changing the lighting conditions of the scene and simply turned the spotlights around and placed panels to obtain diffuse lighting instead of direct lighting. This solved the problem and the vision reached the level of reliability required by the client.
In this post, I would like to highlight an important case of success in the automative industry that has had a relevant impact on its production process, this is the SIVAM5 vision system developed by CARTIF and integrated in cold drawing lines of laminated sheet metal.
As we all know, the surface quality of the vehicle´s exterior is key for users, which is why companies in the automotive sector have to make a significant effort to detect and correct the presence of defects in the bodywork of their vehicles. Most of these defects occur at the stamping stage, but considering the inconsistency of the colour of the sheet metal and the generation of diffuse reflections, in some cases these defects go unnoticed to the body assembly stage and then to the painting stage, after which they become noticeable. This means that a small defect not detected in time translates into a large cost for the production of the vehicle.
To detect these defects at an early stage, we have developed an innovative machine vision system to detect the micro-cracks and pores that are generated in the cold stamping process of rolled sheet metal. This is a clear example of a robust solution based on a simple idea, “the passage of light through the pores of the sheet metal”, but where a great technological effort has been made to implement the idea in the production line. To this end, various optical technologies have been combined with the development of complex mechanical systems, resulting in a high -performance technological solution, capable of carrying out an exhaustive inspection of the critical points of the sheets in 100% of the production and without penalising the short cadence times that characterise press lines.
Thanks to its excellent resistance to vibrations and impacts, its great adptability for the integration of new references and its reliability in the detection of defects, a robust, flexible and reliable solution has been obtained. Based on a simple idea, a robust solution has been implemented in the production process of large companies in the automotive sector, such as Renault and Gestamp, where it has been operating without updates for more than 20 years, working day and night.
There are many research and innovation projects whose objective is the design and development of an electronic device, whose purpose is to satisfy main requirements of the market. In general, we look for devices with the necessary capacity to acquire information about the physical world that surrounds us and, in many cases, interact with it.
To carry out the validation of the idea, it is necessary to carry out a previous prototype that allows a first approximation of the final solution. Generally, the most complex and interesting part is the electronic design of the device. In this part, the design and development of the electronic board is carried out, defining consumption and communication requirements, selecting microcontrollers, PCB board, components, connectors, etc.
This task means to have expensive electronic design software licenses, to integrate expert electronic staff into the work team and to allocate a significant part of the project hours to its execution.
Times change, more and more hardware development platforms are involved in making these changes possible. These platforms offer the user a board that integrates the microcontroller with the circuits and basic components of communication, power, etc. Among them stand out: Parallax, STMicroelectonics, LaunchPad, Microchip ChipKIT, mbed (version of ARM to give solutions to “internet of things”).
But, if I had to choose one of these platforms at this time, I would do it for Arduino. I think he has cleverlycombined the hardware and software, generating a flexible prototyping platform, open source and easy to use, whose features are:
A hardware based on powerful boards that integrate simple microcontrollers. Its main characteristics are low cost, small size and low consumption. It is published under a Creative Commons license, a wide variety of auxiliary equipment developed by other manufacturers that support this platform is available on the market.
Open source software, based on a simple and clear development environment. That allows expert programmers to generate complex solutions. In part, this must availability of a multitude of standardized libraries contributed by a large community on the internet.
These characteristics facilitate and guarantee the integration of the new trends and evolutions that are continuously generated in the field of electronics, thus improving their features and capabilities.
Although a priori it may be thought that this platform is designed to start experimenting with electronics, its features make it a flexible and powerful tool for expert users, facilitating the development of advanced prototypes.
Therefore, these tools allow to reduce costs and design times of any technological proposal, facilitating the creation of prototypes and reducing the errors generated in its development phase. This allows the researcher to forget about the implementation at a low level and focus on the design features.
This technology has great potential for integration in several of the technological research and innovation lines with which the European Union is currently working, such as, the Internet of Things and in Factories of the future, of H2020.
In CARTIF we are aware of its importance and we have started to use these platforms as support in the development of our research work. A sample of this is the European project “SANDS”, where the Internet of Things, Social Networks and Intelligent Systems converge, and the Spanish project “REPARA 2.0”, in which new autonomous and wireless sensors are searched to be embedded in the asphalt layer of our roads.
With this post, I would like to try to show a very clear example where, the intelligent use of a suitable artificial vision system can solve a major problem in a production line at a reasonable price.
The body of our vehicle consists of a multitude of metallic pieces, each with its own requirement. The automotive industry manufactures these parts through a laminating sheet forming process called stamping. In this process a metal sheet is placed on a matrix, it is fixed and later, a punch pushes the sheet towards the matrix generating the desired cavity.
Depending on the temperature of the steel blanks two types of stamping are defined: cold stamping and hot stamping. In this case, we will focus on the hot stamping, which is applied mainly in elements of high structural requirement, such as reinforcements, pillars, etc.
Image captured by the vision system at the exit of the oven
In this process the steel blanks is heated above the austenization temperature, obtaining a high ductility and then proceeding to a rapid cooling to achieve the martensitic hardening of the sheet. The pieces obtained reach high resistance, complex shapes are obtained and the effects of springback are reduced. This allows, among other things, to improve the passive safety of our cars and reduce their weight.
In this manufacturing process, the steel blanks leave the furnace at high speed, at a temperature around 900-950 ºC, they stop abruptly in a fixed position and, later, a robot collects them to introduce them in the press as quickly as possible , In order to avoid its cooling before the press stroke.
The problem arises from the difficulty of ensuring a fixed position with mechanical fasteners. This is due, among other things, to the speed of the line, the great variety of references, the high temperatures of the steel blanks (which cools very quickly at the point where there is a contact) and the internal characteristics of the furnace (which can measure up to 30m).
An incorrect position means that the robot fails to pick up the steel blanks, or worse, to pick it up incorrectly and place it incorrectly in the press, producing a wrong press stroke and stopping the line, together with a deterioration of the tools.
In this case, the artificial vision is presented as the best choice to indicate to the robot if the actual position of the steel blanks is correct. The most important task of the vision system will be to correctly segment the steel blanks into the image in order to accurately determine the position of the steel blanks.
CARTIF position. Application developed by CARTIF
A priori, given the intense infrared radiation emitted by the plates due to their high temperature, it seems that the easiest alternative to achieve this task is to use industrial infrared cameras. This solution presents two problems: the high cost of these equipments and the low resolution of the infrared sensors.
The working area in which the steel blanks are positioned is very wide, due to the size of the parts and because in many cases it is worked in batches, handling up to four units simultaneously. Given the low resolution of these sensors, it is necessary to use several cameras to increase the precision with which the position is defined.
From CARTIF we have been developing more economical solutions, using industrial cameras within the visible electromagnetic spectrum with a greater sensitivity in the infrared range. The resolution of these cameras is much higher than that of the infrared cameras which allows to increase the accuracy of the measurements.
This has allowed companies such as Renault to obtain a robust and configurable system that avoids undesirable stops of the line and extends the useful life of its tools, which leads to a considerable improvement in the production line.
With this post I would like to take up the theme of under road heating, in order to delve a bit more into the benefit that can have heating the most critical points of the road.
As I already indicated, the current solution to avoid and eliminate icing on the roads is the application of chemical deicers, which we all know as “road salt”. To a greater or lesser extent, this substance is sodium chloride, an inexpensive and effective product. I would like to stop here for a little reflection, are we really aware of the damage we are doing using these substances? Surely not, that’s why people rejoice when they see the salt spread.
Millions of tons are scattered annually on our roads, often without proper distribution to the road and with excessive frequency. For this reason, I would like to highlight some of its harmful consequences:
The vegetation near the road is the first to suffer the negative effects of salt, on the one hand, the high concentrations of chloride make it a toxic element, causing the gilding or burning of the leaves, and on the other hand, the High concentrations of sodium can affect plant growth by altering soil structure, permeability and aeration
A significant proportion of the salt is washed away by rainwater reaching aquifers, reservoirs, rivers, wetlands, etc., causing a dramatic increase in the risk of contamination of delicate ecosystems and even in many cases of the water we drink.
Salt greatly affects the health of wildlife from two points of view: due to the serious consequences of its consumption due to its toxicity, especially to birds, and the frequency of run over, since salt attracts the animals for their ingestion.
Another point that we hardly consider is the soil, although its degradation is a serious problem for Europe. Salt reduces the stability of the soil, modifies its electrical conductivity, decreases its pH and in general, seriously impairs its fertility.
As we can see, the environmental impact of chemical deicers is very intense, therefore, we should try to make an effort to minimize their effect, using all the technology that is within our reach to achieve a less aggressive winter maintenance
A partial solution would be to be able to measure in real time the amount of chemical deicers at each point of the road, not just at a fixed point. This would only be achieved by loading the sensors into the intervention and maintenance vehicles. Currently, there are some systems under development that measure wheel splatter, measuring the water refraction index (Japan Highway Public Corporation) or electrical conductivity (University of Cone). Given their results, they have never been incorporated into the market.
From CARTIF, with the collaboration of the Spanish company Collosa, we are investigating in the development of this product. The objectives are to avoid spreading more road salt when the current quantities are sufficient, to throw only the necessary quantity in the precise place that needs it (given the system of global positioning of these devices) and to give an objective tool to the responsible of the winter maintenance, so that he can make the right decision.
In CARTIF we are committed to a final solution that avoids dispersing chemical deicers as far as possible. If we manage to attack the problem in the most dangerous points, preventing and avoiding the formation of ice, we will avoid the exit of the truck to cover those points with chemical deicers. In addition, this outlet will not only cover the dangerous points, but will spread the chemical deicers all over the road.
This solution is the development of a more economical radiant floor with more energy efficiency, based on geothermal energy. For this, the development of an intelligent prediction that prevents the formation of ice and is based on the use of new bituminous mixtures is fundamental.
Undoubtedly, this will mean a significant reduction in the environmental impact of winter maintenance on our roads and in particular in the most sensitive areas of our geography such as natural parks.
