Actuated Textile Hybrids

This paper presents the prototyping of new methods by which functionally graded materials can be specified and produced. The paper presents a case study exploring how machine learning can be used to train a model in order to predict fabrication files from formalised design requirements. By using knit as a model for material fabrication, the paper outlines the making of new cyclical design methods employing machine learning in which simpler prototypical materials acts as input for more complex graded materials. A case study - Ombre - showcases the implementation of this workflow and results and perspectives are discussed.


1) INTRODUCTION
Contemporary building culture is finding itself in an era of resource scarcity.The demand for new buildings to serve expanding populations across the world is creating new pressures on the materials needed for construction such as sand and aggregate for concrete as well as many basic minerals (Valero et al. 2010).This challenge is inciting research into new material practices that optimise resource allocation and reduce waste (Ramsgaard Thomsen et al. 2019).By moving from the paradigms of standardisation and mass production, these new practices position the making of hyper-specified and functionally graded materials for site-or use specific application as central means of building smarter with less.
This paper discusses the prototyping of new methods by which such complex heterogeneous materials can be specified and produced.The paper presents a case study -Ombre -exploring the cre-ation of predictive models in which performance at high scale can be interfaced with low scale specification data.The case study examines how simple prototypical materials can act as input for the specification of more complex graded materials.By using knit as a model for material thinking, the paper speculates on future fabrication paradigms for a lighter architecture.
The enquiry was first examined in a workshop with Masters students from CITA Computation and Architecture.

2) THE SPECIFICATION OF KNITTED TEX-TILES -A MATERIAL DESIGN PRACTICE
The enquiry builds on CITA's extensive practice with knit as a material system (Ramsgaard Thomsen et al. 2008Thomsen et al. , 2012Thomsen et al. , 2016)).Knit is a highly versatile fabrication system.The composition and resulting performance of knitted textiles can be minutely controlled By working at the stitch level, knit allows us to design materials at micro level.Knit is inherently an additive fabrication system in which the textile is produced "to form" thus reducing waste rather than mass produced and then trimmed (Ramsgaard Thomsen et al. 2008).As such, knit is an interesting model for a broader material practice of hyper-specification.
In CITA we have examined the employment of knit as a material for architecture (Ramsgaard Thomsen et al. 2015, 2019b).As part of these enquiries, we question how performance requirements can be formalised and interfaced with material specification.Former work examines two central methods: 1) simulation -in which performance is simulated and then interfaced with fabrication files resulting in shaping of knit patterns, defined detailing as well as patterning allowing graded performance across the membrane and 2) sensing -in which local sense data is interfaced with knit fabrication files enabling local transformation of pattern structures.This enquiry extends these investigations by probing how imagebased sense data can be used to train a model by which fabrication files can be predicted.In Ombre, the knitted textiles are understood in respect to their capacity to filter light.By using a set of prototypical textiles as base information, the model learns to map particular light intensity requirements to particular knit patterns.The learnt mapping can then be used to generate fabrication files from formalised performance requirements.

3) GENERATIVE ADVERSARIAL NETWORKS
In Ombre we apply Generative Adversarial Networks (GANs) to predict knitting patterns.GANs (Goodfellow et al. 2014, Goodfellow 2016) are a type of deep artificial neural network, a particular approach within machine learning that has driven the majority of recent progress on artificial intelligence across academic and applied industrial application.Deep neural networks, which have multiple layers of perceptrons, are particularly well suited to visual tasks and have been applied to detecting and classifying objects in images, edge detection and semantic segmentation (Schmidhuber 2015).GANs have been particularly applied to the generation of new data, including the generation of videos and images (Huang 2018) and medical images (Han 2019).This makes them particularly interesting for architecture, as they address the problem of representation (Gero 1991) by allowing design knowledge to be represented and generated in the form of pixel images -a common representation within the architectural design process.Within architectural research, deep neural networks have recently been applied to predict forming tolerances for robotic metal fabrication (Nicholas et al 2017, Rossi et al 2018), and GANs to recognise and generate floor plans (Huang et al 2018a), and for style transfer (AITIME 2019).
Where traditional architectural models are explicitly represented, captured via rule-bases or solved via calculation, machine learning models are predictive.The fundamental goal of this modeling approach is to generate new information by generalizing beyond known examples (Domingos 2012).GANs are trained on a dataset of existing images.They learn to imitate the distribution of information within this dataset, which enables the prediction of unseen distributions post training.Training is achieved via an adversarial process that combines two different neural networks -a Generative model that makes instances of new data, and a Discriminator model that evaluates data for authenticity.During the training process, which typically runs to several thousand iterations, the Generator trains to gen-erate passable data instances, while the Discriminator trains to classify images coming from the Generator as fake.In the pix2pix architecture (Isola et al. 2017) we have employed, training data is given to the Discriminator in pairs, and the parameters of both models are adjusted over time until the discriminator is unable to distinguish generated from original data.After training, only the generative model is required to create new images.

4) THE DESIGN SETUP: WORKFLOW FOR PREDICTIVE MODELLING OF MATERIAL PERFORMANCE
The enquiry devises a simple circular workflow in which an iterative design process is formed.A first set of simple prototypical patterns are devised as training data for the algorithm.The patterns are knitted and their resultant shadows recorded under a robotic artificial sun allowing us to locate the material and use precise sun paths.For this enquiry we located the material in Copenhagen, noon, 21st of June.

The knit file
The present enquiry employs single knit jacquard as knit structure.Jacquard knitting allows us to knit complex patterns with continuous lengths of yarn controlling each stitch individually (Spencer 2).As the knitted structure foregrounds a chosen yarns, the other yarns 'float' behind the fabric.This allows us to control the pattern locally.However, the relationship between the fibres is not binary.Depending on the chosen yarn types, the floats affect material performance both structurally and visually.

The fabrication environment
Jacquard knit is based on the same punch card logic as jacquard looms (Spencer 1989).This means that a binary logic controls the knitting machine and the changing of the yarns.In early mechanical knitting machines long paper tapes were used to define patterns on the stitch by stitch level.From the outset, these first computational codes were used for custom material specification.Ombre is knitted using a domestic single bed knitting machine (Brother KH910) with a electronic control.In their original state these machines are preprogrammed with 900 standard repeat based patterns.By using an AYAB shield (Smith 2016), we are able to replace that standard hardware with an open interface that allows us to define our own patterns.This allows us to control the knitting machine from the computer employing binary custom bitmap.The bitmaps are composed of white and black pixels each representing the two different yarns being used.
In Ombre square samples (40cm*40cm) are produced from the bitmap input images (130px*215 pixels).A scaling factor is applied to achieve target di-mension of the knit samples.This scaling factor derives from preparatory tests , where the relation between several knitted samples and the corresponding bitmaps is calculated.
In the design-to-fabrication workflow multiple parameters affect the resulting shadow.Yarn thickness gives a particular density to the knitted surface.Ombre is knitted with two highly different yarns; a white polyester spun to create a mechanical stretch (0.3 mm) and a black polyester wrapped with a copper foil (0,1mm).When the two yarns are combined in the jacquard pattern interesting material behaviour is observed.Areas of predominant white elastomer become dense as the yarn contracts

Sensing light
In order to evaluate the knitted samples we use the industrial robot as an artificial sun.By attaching a parallel light source to the robot arm we steer the movement of the light based on sun path simulations.The setup simulates the 1:1 material probe in a real world light context.For this enquiry we use weather data files (.epw) for Copenhagen as the basis for the sun path simulation and employ Ladybug for Grasshopper to get the robot tool path.In order to switch the artificial sun on and off we use a relay triggered by bi-nary information sent by grasshopper directly to the robot.We use the low power voltage output from the robot (24 volt) to trigger the relay and switch on and off the high-power light source (230 volt).In this way, we can align the digital simulation (grasshopper) with the physical simulation (artificial sun).
The sample is attached to a jig which is placed under the robot.The jig is a simple adjustable frame which can hold the knitted sample in front of a translucent plane which is used as a shadow receiver.The jig allows multiple samples to be compared in a fixed experimental setting.A camera is placed under the translucent plane to record the shadow movements and the qualitative change of the filtering of light.The shadow images, together with their paired fabrication files, are used as the input for the teaching of the neural network.Additionally, the shadow's qualitative and quantitative effects are used to steer further design iterations.

4.4) Design iteration
The first iteration of prototypical knitting patterns and corresponding shadow effects of the resulting knitted textiles act as the basis for training our neural network.The three designed patterns represent "extremes" of the knitting pattern design space.The extremes are: a) half of the pattern image is white and the other half of the image is black, b) all pixels in the image are alternating between black and white and c) pixels change in overlaying radial gradients (for fabrication translated into dithered black and white image).

4.5) Data processing
We use the shadow images of the knit and the fabrication files to train the neural network.
and to generate the fabrication file based on a desired top-down designed shadow effect.
The training data for the neural network is based on image samples of the knitting fabrication files (training output) and on shadow images from the produced knitted textiles (training input).To relate the knitting fabrication files to the shadow images of the knitted textiles, the fabrication files are scaled and stretched to have the same proportions and image size (1536 pixel by 1536 pixel).
In order to generate a dataset of suitable size and variation to teach the neural network a generalised relationship between fabrication file and shadow effect, we use image crops of the three original fabrication and shadow images.The original images are cropped into overlapping samples of 256 pixel by 256 pixel.This way we create an input and an output dataset with 441 samples for each image equating 2646 input and output samples in total.This way we can create a meaningful relationship between the knitted material and the fabrication file based on only three knitted samples.The training input and output samples are merged together for the teaching of the neural network.
After teaching the neural network we use the three original shadow images to evaluate the network's prediction accuracy.The input images are chopped into samples, run through the neural network and merged back together into large images afterwards (1536 pixel by 1536 pixel).Through the testing of different training iterations of the network (epochs), we found that training the network for for 80 epochs was sufficient enough to deliver good results for all three original image pairs.The training was run for approximately 10 hours on a GeForce 950MX using Nvidia's Cuda Technology.
The neural network learns to adjust its weights for all three extreme shadow design cases.Following this process we use the trained generative model to make predictions of fabrication files based on desired shadow effects.In the final design, we create a gradient image (a) which we use to define desired shadow zones (b), which is then stamped with shadow textures (c).We use the texture image, crop it into small images of 256 pixel by 256 pixel to run it through the network.The learned output images are merged back together (d), scaled back to the original proportions of the knitting file (e) and dithered into black and white pixels (f ).The dithered image is used as the input for the knitting machine and we produce the final learnt knitting sample (Fig 10).

5) DISCUSSION
What has been presented here is a prototypical method.The case study, Ombre, proves that the workflow of training a model to predict fabrication based material specification is viable and can allow more intuitive approaches to realising design intent.In Ombre, the single jersey jacquard patterning and complex interchanges between the two fibres and the floats create unpredictable effects that can not readily be mapped.Here, the predictive model assists us in driving the fabrication data.The implementation of a GAN to solve this enquiry is interesting because it is image based and multiscale.In Ombre, both sense data and fabrication files take the form of image files.Furthermore, the underlying multi-scale architecture of the algorithm and the way that smaller subsections of images are parsed with larger sections is akin to the inherent problem of material specification in which performance at high scale requires interfacing with low scale specification data.At present some parts of the workflow are still formative.We can still only predict within a certain distance from our training data and the process of generating the input file -the stamping of the image file -is still place holding a more generative approach.

5.1) Limitations
The most immediate limitation in this enquiry is that all training is undertaken on the basis of a relatively few still images captured at a single time and date.The opportunity of the artificial sun is the ability to record multiple light-states and therefore train the model across multiple performances.As shadows move the mapping between input and output changes.Further steps are needed train the model across multiple time-steps.A second limitation is the choice to work with relatively two dimensional textile sample.In early tests and during the workshop we experimented with more three dimensional textiles which have larger impact on the transformation of shadow patterning over time.The case study is a limited enquiry and further work is needed to evaluated the precision of the trained model across multiple samples with different strategies of differentiation.

5.2) Perspectives
The opportunity of the workflow is to enable a designerly and more intuitive specification of materials.The designer does not need to have in-depth understanding of how the stitch patterns affect and change performance.Furthermore the opportunity is to scale up in complexity.Where Ombre is trained on a set sample size the model can be implemented on larger arbitrarily sized and shaped patches which can otherwise be unmanageable and excessively time consuming.Finally, the opportunity is also to transfer these specification strategies onto different kinds of performance criteria.Where Ombre performs through light filtering thereby creating a direct and easily understandable correlation between sense data and performance, further work could transfer these performances into other domains such as structural or thermodynamic performance where simulation is difficult and empirical sense data available.
Figure 1 Complex material specifications in knit, workshop CITA Computation and Architecture Figure 2 Circular design workflow for informed design iteration.
Figure 3 Float system in single jersey jacquard and resulting knit structure Figure 4 AYAB interface, binary fabrication file and laser cut punch cards.Laser cut punch cards were used in early test to understand the binary logic as well as horizontal and vertical repeats.
Figure 5 Robotically steered artificial sun and jig Figure 6 Knit fabrication files A, B and C: resulting knitted textiles and adjusted shadow maps Matter -MATERIAL STUDIES AND INNOVATION -Volume 2 -eCAADe 37 / SIGraDi 23 | 491 Figure 10 a) Gradient image file describing design intent of varying shadow performance (white pixels -white elastomer, black pixels -copper yarn), b) shadow zones defined across 6 gradients, c) shadow zones stamped with shadow textures for training d) learned output, e) scaled learned output, f ) dithered fabrication file Figure 11 Resultant knitted fabric from learnt fabrication file