Re-perceive 3D printing with Artiﬁcial Intelligence

How can machine learning be combined with intelligent construction, material testing and other related topics to develop a new method of fabrication? This paper presents a set of experiments on the dynamic control of the heat deflection of thermoplastics in searching for a new 3D printing method with the dynamic behaviour of PLA and with a comprehensive workflow utilizing mechanic automation, computer vision, and artificial intelligence. Additionally, this paper will discuss in-depth the performance of different types of neural networks used in the research and conclude with solid data on the potential connection between the structure of neural networks and the dynamic, complex material performance we are attempting to capture.


BACKGROUND
After years of extensive development, artificial intelligence has proven its potential to achieve humanlevel performance in many areas such as image recognition, speech recognition, and decisionmaking. This technology can be used in a far broader direction than the abovementioned fields. Currently, various fields are exploring how to use artificial intelligence technology to solve professional problems. In the field of intelligent construction, architects continually explore material understanding and construction methods.
In the field of digital construction, the application of digital methods to traditional building workflows has always been one of the important research topics in the construction industry. The development of automation technology, especially the emergence of various machine manufacturing, has brought new possibilities for the construction industry. In material development and use, progress in digital fabrica-tion has benefitted from incorporating dynamic material properties as generative and integrating design factors (Menges, 2012a). Architects and construction builders can gradually transfer materials to the digital manufacturing field. Numerous studies have been conducted to digitize craftsman's traditional work, ranging from utilizing digital fabrication tools to simulating the handiwork of handcrafted items (Brugnaro and Hanna, 2018) to augmenting actuators with sensors, to incorporate the real-time feedback on a material's state during the design production process (Mueller et al., 2012).
Contrary to the human understanding of material performance based on explicit material properties and behavioural models, neural networks are able to fit any function allotted enough training materials without human understanding of the logic behind it. Therefore, is it possible for us to unite artificial intelligence (AI) and mechanical automation processes to generate an end-to-end material per-formance model that bridges fabrication control and outcome of the form while utilizing complex and dynamic material behaviours without a scientific understanding of the material properties? We will offer the best possible exploration of this issue.
Recently, research has been conducted to use AI to develop a novel fabrication method. For example, neural networks (NNs) have been explored to develop design tools for doubly curved metal surfaces (Rossi and Nicholas, 2018), and in other cases, various robot woodcarving techniques have been assessed in response to material properties (Brugnaro and Hanna, 2018). This research will potentially give architects and designers a universal method to access a diversity of material properties and enable designers to program material behaviour and achieve design intentions without systematic understandings of the mechanism behind material performance.

PREVIOUS RESEARCH
This research is a continuation of a previous method that utilized robotic automation and AI to observe the complexity of flexible elastomer materials in regard to their bending and deformation features(Luo, 2018). The experiment described in this paper is developed to seek more insight to the questions raised from previous studies: 1. Is this a universal method that manages material properties and has no established material model? 2. Instead of mapping material distribution/properties to material behaviour, can this process generate a more abstract end-to-end model that directly maps mechanical control of the fabrication tools to the behaviour of material and the final form? 3. How will this method impact common fabrication technologies?

OVERALL SYSTEM
In this research, we are applying the method of automatic generation of the material performance model with AI to one of the most common fabrication technologies-3D printing with PLA.
We have developed a system for a printer to print a line across two supports at either end and to let the material naturally drip into a curve. As a thermoplastic material printed in such an unsupported manner with alternating extrusions, movement speed determined by Gcode, and under the compound play of gravity and intermolecular forces, the PLA string will solidify into a curve with uneven curvature. This system is different from the traditional additive layering printing system. Conventional printing systems require layer-by-layer printing from bottom to top, and additional brackets are required for some of the suspended areas. The system is capable of efficiently printing a 2.5D or 3D spatial wireframe without additional support. Owing to the complexity of intermolecular forces, unknown material properties, etc., planning and control of the printing path is impossible to achieve with current 3D printing software, as this 3D printing method relies on the dynamic heat deflation of PLA that has no established method for simulation or prediction. Confronting such a challenge, a machine learning approach is developed to build an end-to-end workflow. After collecting a large number of training samples with an automatic system and image processing, we created two models to learn the rules for this new printing method: One forward model calculates the printed curve via a given Gcode, and one backward model predicts the Gcode controlling the printer by producing the curve that the designer intended to print (Figure 1).

Figure 1 Two-way mapping between Gcode and the printed form
The main workflow of the method includes the following steps.

Figure 2
Image to data age, extracting the data describing the material performance with image recognition, matching the fabricated form with the Gcode that controls the movement of the extruder and formatting it for machine learning.
In the experimental design, we printed white PLA materials against a dark background for maximum contrast. In this way, we can easily extract the experimental objects and transform the pixel points into sequenced data according to the requirements of subsequent experiments with image processing by Python and Opencv. At the same time, the curve data generated from the image is matched with the corresponding printed Gcode, forming a complete training sample (Figure 2).

2.Training the forward and backward models.
We use the deep neural network model in machine learning to build a model for predicting curves from Gcode and a model for predicting Gcode from curves, using appropriate data augmentation techniques. By selecting different types of models, different model structures and different hyperparameters for comparison, the best-performing models are finally trained for the subsequent experimental sessions. Information about neural networks and specific models will be detailed in the next section.
3.Model and method evaluation. The trained model was used to predict the Gcode from a given curve, which was printed and compared to the deviation between the predicted curve and the actual print to evaluate the ultimate effect of the method.

Deep learning
Traditional machine learning and signal processing techniques explore shallow learning structures that contain only a single layer of nonlinear transformation. A commonality of shallow models is that they contain only a single simple structure that converts the original input signal into a particular problem regarding a spatial feature. The concept of deep learning stems from the study of artificial neural networks, which combines low-level features to form more abstract high-level representations (attribute categories or features) to discover distributed feature representations of data. In 2006, Hinton proposed a deep neural network for complex general learning tasks in science, pointing out that networks with a large number of hidden layers have excellent featurelearning capabilities (Hinton, 2006;Yoshua 2009;Arel, 2010). Humans have found a way to deal with "abstract concepts" with the help of neural networks. The research of neural networks has entered a new era, and deep learning has begun to enter a period of rapid development.
In this paper, three models-neural networks (NNs), convolutional neural networks (CNNs) and long short-term memory (LSTM)-were tried in the experiment. By comparing and analysing the performance of the three models, the best performing model was selected and utilized in subsequent ex-periments. In choosing a loss function, we used mean square error (MSE) and mean absolute error (MAE) evaluation methods. The lower the values of MSE and MAE, the smaller the model error and the better the performance.
NN is the simplest deep learning model, and for some simpler problems, it is the most efficient solution. CNNs can significantly reduce the number of free parameters in the network. A small portion of the image referred to as the localized region in the CNN serves as the bottommost input to the hierarchical structure. Information is passed through different network layers, so each layer can acquire significant features of the observed data that are invariant to translation, scaling, and rotation (Erhan , 2010)( Figure 4).
A RNN has a cyclic network structure and the ability to maintain information. Its network structure is shown in Figure 5 The cyclic network module in the RNN transfers information from the upper layer of the network to the next layer. The output of the hidden layer of the network module depends on the information of the previous moment. The LSTM network is an extension of the RNN and is specifically designed to avoid long-term dependency issues. The LSTM's repetitive neural network module has a different structure, and unlike the plain RNN, there are four neural network layers that interact in a special way(S. Hochreiter, 1997).

Data augmentation
In machine learning, sometimes there are not enough training samples, or the sample distribution is not balanced, which may lead to poor training results and overfitting. To solve these problems, it is necessary to generate some new samples with data augmentation, on the premise of ensuring accuracy on the basis of existing samples, and expand the training per se. In the process of collecting samples, we used certain consistent characteristics of the printer and image processing to conduct data augmentation.

Forward model
For the three models-NN, CNN, and LSTM-the comparisons of different amounts of input data and different model structures are shown below.
For the three models, the overall performance of the LSTM model is significantly better than the NN and CNN models. For sample input data of different sizes, the greater the number of samples as a whole, the better the model performance; this reflects that the deep learning model has a strong dependence on the size of the training samples.
However, in the case of 1100 and 1600 training samples, the performance of the model has its advantages and disadvantages. In response to this phenomenon, we have conducted further research. We train all the samples in the training set, but for the test set, we divide it into subsets according to the different amplitudes of the curves and separately evaluate the prediction.

Figure 6
Performance of forward models It can be concluded from the above figure that the performance of models for different amplitude intervals is significantly different. For the three models, the amplitude is predicted most accurately in the 10-20 mm range, and the more deviation there is from this amplitude range, the worse the model becomes at prediction. In the amplitude range of 10-20 mm, the average prediction error of each coordinate is only approximately 2-3 mm, which indicates that the Backward model performance over difference amplitude ranges method is very feasible. Selected testing samples are shown in Figure 8. One possible explanation is that in the process of collecting samples in the early stage, the distribution of the samples has a certain deviation. At a higher sample distribution with an amplitude closer to 10-20 mm, the prediction effect of the interval is obviously better than other intervals. A solution to this issue can be further improved upon in subsequent studies.

Backward model
In the previous experiment, we trained the model to predict the shape of the curve with a given Gcode. However, in practice, what is more commonly needed and has more practical value is a model that predicts a possible Gcode from a given targeted curve.
Similarly, for this backward model, we continued with the three models of NN, CNN, and LSTM, as well as different input sample data volumes and different model structures. The results are given below ( Figure  9).
For the predicted Gcode model, the differences between the different models are not as large or even significantly different from those of the predicted curve model. However, as the number of input samples increases, the backward prediction accuracy of the CNN and LSTM decreases significantly more than the NN. This aspect shows that we are do not have enough samples to sufficiently train the backward model. On the other hand, it also shows that the NN is more adaptable to reverse prediction.
Similarly, we use a full sample to evaluate the performance of the inverse prediction model over different intervals (Figure 10). In the 20-30 mm range, the NN can still maintain a good level of accuracy compared to the CNN and LSTM. Therefore, the NN is more suitable overall for the reverse prediction model. Selected testing samples are shown in Figure  11.

METHOD AND MODEL EVALUATION
Returning to the purpose of this research, we hope to propose a new manufacturing process based on dynamic material properties. In terms of the perceived material properties, our model has already achieved a positive result. For a given set of Gcode, printed shapes can be predicted with a certain degree of accuracy. Considering the actual production value, however, we still have to focus on whether we can predict a possible Gcode set for the curves intended for printing.
Therefore, we will conduct a set of experiments to validate the trained model. A simple landscape scheme is put together and divided into a set of cutout curves. The trained model is used to predict the Gcode for printing. We print out the Gcode and compare the actual print results to the expected curves to evaluate the effects of the method and model.
The actual print results are basically consistent with the expected curves (Figure 12), which proves the feasibility of the method.

CONCLUSION
The conclusions of the paper can be summarized as follows: First, the method and outcome in this paper establishes a novel method for 3D printing of spatial wireframes and its development process with an automated machine learning process. Second, it has expanded and generalized the potential application of the proposed workflow beyond the limitation of current scientific material models. Third, such a method is capable of creating a direct end-to-end connection between the control of the fabrication process to the final form resulting from a complex form fining process, avoiding complicated interim steps and calculation. Last but not least, compared to the traditional method of developing material models that require different fields of knowledge and workflow for different material properties, the method and logic of the work described in this paper is universal and proven to be capable of generalization when applied to producing a diversity of material performance models that encompass multiple systems of non-related ma-terial behaviour, such as the active process of bending elastomer and the reform process of melting thermoplastics.
For architects, architectural design ultimately relies on the selection and construction of materials. In the long run, new building materials and new construction methods will bring about tremendous changes in the construction industry, as well as new architectural styles. We believe that the approach presented in this article will be a positive inspiration to herald this change.