Machinic Interpolations: A GAN Pipeline for Integrating Lateral Thinking in Computational Tools of Architecture

In this paper, we discuss a new tool pipeline that aims to re-integrate lateral thinking strategies in computational tools of architecture. We present a 4-step AI-driven pipeline, based on Generative Adversarial Networks (GANs), that draws from the ability to access the latent space of a machine and use this space as a digital design environment. We demonstrate examples of navigating in this space using vector arithmetic and interpolations as a method to generate a series of images that are then translated to 3D voxel structures. Through a gallery of forms, we show how this series of techniques could result in unexpected spaces and outputs beyond what could be produced by human capability alone.


INTRODUCTION
The transition of architectural tools from traditional to modern computational (Claypool, 2020;Steenson, 2017) has focused on efficiency and productivity.This is so as many of the computational tools used by architects have their legacy in industrial disciplines such as automobile and manufacturing (Claypool, 2019).CATIA, for example, a software used by Frank Gehry and his team to achieve complex designs, was initially developed for the aeronautical industry (Claypool, 2020).
As a result of being inherited from these fields, the values and standards of these other disciplines have discretely trickled architecture.(Claypool 2020;Steenson, 2017) Consequently, notions of optimization, standardization and efficiency, all qualities of vertical thinking, or industrial era reasoning, are inevitably prioritized in these tools over lateral thinking qualities like inaccuracies, uncertainty and accidents, qualities often associated with artistic practices.
Yet, as the work of academics such as Bryan Lawson, Nigel Cross and Kees Dorst states, there is a clear relationship between creativity and design excellence.(Lawson, 2006, Cross, 2006, Dorst et al. 2001) Their work points out that successful architectural design must have creative design processes and that this creative process is manifested in a designer's ability to balance divergent thinking and convergent thinking -with divergent thinking being closely related to "lateral thinking", the terms coined by Edward be Bono.(De Bono, 1985;Lawson, 2006,) Such a creative design process is what often brings about unexpected design opportunities.(Lawson, 2006, Cross, 2006, Dorst et al. 2001) Hence, as our tools became more and more digital, the features and methods imposed by these tools, which greatly focus on vertical thinking, have automatically and inadvertently constrained the possible design space.(Claypool 2020) In this paper, we target the need to re-integrate lateral thinking strategies in architectural design tools.We argue that today's computational tools that lack these specific qualities could benefit from embedding artificial intelligence at the early stages in the architectural design process as a method to address lateral thinking approaches in our processes.Specifically, we present a 4-step AI-driven pipeline, based on Generative Adversarial Networks (GANs), that draws from the ability to access the latent space of a machine and use this space as an experimental digital environment to design 3D architectural structures.

RELATED WORK
Architectural tools for the last decade or so have mostly focused on parametric approaches.These approaches are characterized by the assignment of values to relationships between elements and manipulation of those values to control and organize these relationships.On the other hand, machine learning algorithms work by exploring data and identifying patterns within them without relying on predetermined equations as models.Several architects and researchers have recently explored the potential of machine learning algorithms, particularly Generative Adversarial Networks (GANs), in improving creative possibilities (Huang et. al 2018;Chang et al 2020;Chaillou 2019;Zhang 2019;Newton 2019).These GAN algorithms are able to statistically learn and imitate any distribution of data presented to them and are commonly used for generative purposes.et al. (2018) applied GANs in recognizing and generating architectural drawings.The authors used pix2pixHD, a modified GAN that learns image data in pairs and generates new images based on the input.They used diagrams of rooms marked in different colors as input to produce apartment plans.Chang et al. (2020) proposed a graph-constrained generative adversarial network to automatically generate multiple house layout options from architectural bubble diagrams.

Huang
These works clearly demonstrate the potential and ability of GANs to generate architectural plans and possibly optimize architectural processes and workflows.The results are accepted based on how familiar or similar to the current state of architecture they were.Hence, in terms of lateral thinking approaches, these methods do not perform well as their output tends to always be rather predictable, expected and constrained to what is already known.
Yet, as Philippe orel states in his article, "the task of any architect is not about using computers to replicate or to automatize what has already been thought of and produced, it is about allowing computers to reveal a fully new form of architectural intelligence."( etsin, 2019) He elaborates that " achines, ... shall logically give birth to a kind of architecture that is also beyond our usual capacities", and states that such an architecture is still to be produced.( etsin, 2019) Our work aims to expand on this and allow the machine to reveal a new kind of "unthinkable architecture."( etsin, 2019)

U PREDICTABLE OUTPUTS
Chaillou (2019) proposed an approach of using nested GAN models to generate an entire apartment building.
sing this method, the author demonstrates how nesting different models results in unique outputs and new architectural styles that fall in-between currently known styles.Although such an approach reveals great results, the process still involves the heavy curation of image pairs to train on.Zhang (2019) explores the use of another GAN that does not rely on image pairs.This network, called StyleGAN, is used to mix and interpolate between different types of 2D input data to generate 3D forms.By mixing between satellite images, styled plans and sections, the author demonstrates how the generated 3D forms are only possible to conceive with the use of such a tool.Newton (2019) provides a survey of GAN technologies and reveals a series of potentials of such technologies in architecture by showing experimental results of 2D drawings, such as plans or elevations, or 3D forms, generated with voxels.
Our paper builds on this body of work and proposes a series of steps and approaches through which we discuss the potential of AI-driven lateral thinking in our architectural tools.

T ODO O
We propose a pipeline (Figure 1) to generate 3D structures that uses StyleGAN and explores its latent space as a 512dimensional virtual environment to design in.A latent space is an n-dimensional space where, after training a GAN model, the generator of this model learns to map points in this space to specific generated images similar to the ones of the dataset it was initially trained on.(Karras et al., 2019) These mappings change every time the model is trained.
The latent space has a structure that can be queried and navigated where every point in this space is a 512D vector that represents an image.Our 4-step process draws from the abilities in and properties of this space to design a 3D structure.(1) It consists first of the initialization of this virtual latent space of points through the training of the StyleGAN model.(2) The following step is the entry into this space and access into its dimensions by choosing or generating a 512D vector that represents an image.(3) Further, after getting a point (an image), we navigate through the space by using vector arithmetic operations and interpolations to generate new points that are translated to images.(4) Finally, we generate a voxel 3D structure by using the pixel values of these images.We showcase this pipeline and suggest how it can push architectural digital tools into the grounds of experimentation.
The virtual environment represents the digital design space in which architects currently design, analyze, edit and ector representations e ifferent metho s of interpolation in this space f o eli e geometries pro uce from se uence of images generate generate 3D structures.The current tools used, such as hino3D, focus on orthographic approaches and rely on curves, lines, surfaces, N BS, etc. and the manipulation of those elements to create designs in a 3D virtual space.In this paper, we initialize instead a 512D virtual design environment through the training of a machine and propose different approaches of designing within that space.

I ITIALI I THE IRTUAL E IRO E T
To initialize the virtual environment, a StyleGAN model is trained on 5000 architectural plans scraped from several different online sources such as ArchDaily (ArchDaily.com) and Artstor (Artstor.org).The scraped dataset consists of mainly housing plans of different styles and periods.This allows for better control over the possible image features that the machine might learn.After training, a latent space of 512D is generated where data points (which represent images) with similar features are placed closer together in this space.We use this generated "virtual environment" as a search and a design ground for design possibilities.

E TERI THE SPACE
The following step is the entry into this space.We access the dimensions of this latent space using two main methods through point generation, by randomly using a set of values for a 512-dimensional vector (an image) or through image embedding of a drawing chosen or designed by the user.
In the first method, the machine goes through the latent space and generates an image that corresponds to the specified vector values.In the second method, a StyleGAN embedding algorithm can be used wherein the machine maps the embedded image to a specific point in the space closest to images that have similar features.

A I ATI THE SPACE
After generating or selecting points in this space, we use these points in vector arithmetic operations to create new points that are then translated to images.To illustrate, by adding vectors, we combine different points in space that have different features and generate a new point.Specifically, in terms of architectural plans, by adding different points (image vectors) in space, we are adding vectors that correspond to plans of different attributes and possibly styles in architectural design, depending on what the initial dataset is.This allows for the targeted yet unpredictable generation of images.Figure 3 shows one of these outputs in our process.
oreover, apart from combining vectors, we interpolate linearly between two or more points.By interpolating from one point to the other, we create a series of transformational, or transitional, images with different number of steps or degrees of similarity, as illustrated in Figure 4.

E ERATI D O ELI ED STRUCTURES
Finally, to create the 3D forms, we stack the series of images generated in Z-space and generate a voxel structure.To do this, in this step, the pixel values of the interpolated images are read and translated to a 3D array of Boolean values which define the solid voxel volume.
This virtual environment (latent space) generated serves as an alternative space to the environments of our current computational tools such as hino and AutoCAD.Working within the space of a machine imposes new biases of its own.Yet, we argue that the current "mechanistic" operations usually performed in the current 3D modeling software, such as slicing, lofting, trimming, etc., produce relatively predictable output by following a series of steps and rules.
However, through our integrated non-sequential and nonlogical pipeline and latent space directional navigation techniques, we aim to escape from the established methods of manipulating curves, lines and surfaces of our tools and move toward working with vector arithmetic operations and pixel and voxels as building blocks as a lateral thinking approach.Such an approach may open new opportunities to design and inspire new spatial and formal concepts.Additionally, in this process, the machine acts as an active player to the design output rather than as merely a means to an end.We demonstrate this in the following section through a series of unique 3D forms generated using 4 initial starting points in the latent space.

R U T
To test our pipeline, we select 4 random points (images of plans A, B, C, D) in the generated latent space and get their vector representations.We then test 4 different methods of interpolating and navigating between those points and generate a total of 200 images that we later use to generate 3D voxel structures.

A I ATIO ETHOD E UAL RATIOS
In the first approach, we test the effects on the generated form of changing ratios between 2 plans during interpolation.As a first test, we give an equal opportunity to both plans (50 50).

A I ATIO ETHOD U E UAL RATIOS
In the second approach, we change the ratios of the points in each interpolation.By changing ratio, we give a higher percentage to one of the images to generate more similar images to itself than to the other one.

A I ATIO ETHOD I I
For the third approach, we find the average vectors of the 2 points between which we are interpolating and interpolate between those new vectors.A 50 50 interpolation between images is done in this step.In Figure 3, we show the results of mixing with different percentages of each plan image.

A I ATIO ETHOD CO BI I I I U E UAL RATIOS
Finally, in the last approach, we combine the last two methods mixing and unequal ratios interpolations.
In Figure 2, we demonstrate the performance of the four approaches presented.The order of interpolation is from A to B to C to D and 50 steps are taken for each interpolation.

CHA I ORDER
However, interpolating in this order is ust one of the possible ways to do so.We visualize, in Figure 5, the results of interpolating in different orders between points and show how this simple drastically affects the generated output.
re orms generate using ifferent sets of initial points re Generate forms using the same points ut ith ifferent se uence of interpolations

CHA I POI TS
Furthermore, all these 3D structures are the result of interpolating between four chosen points.Yet, in such a wide latent space, we have an infinite number of options to interpolate between.Figure 6 shows a gallery of outputs generated with different sets of 4 initial points.

DI CU ION
The use of GANs and its internal properties in this paper presents us with a new set of interaction methods that normally lie outside the traditional boundaries of the architectural discipline.By incorporating these discrete steps into our processes, we demonstrate an alternative experimental space where form is indirectly generated through a series of different sequential 2D images.We argue that using the pixel and the voxel as the building blocks of architecture can free architectural drawings from tools of precision and also retrieve a drawing's potential for generative, lateral thinking.Although generated from architectural plans, the resulting highly complex gallery of forms (Figure 5, 6) fall at the intersection between conceptual models and architectural structures, portray distinctive aesthetic features and transcend the architectural outputs we are familiar with today.
Although the forms are derived from 2D plans, this methodology is not meant to be the sole driver for an architectural pro ect.Also, after generating the 3D structures, the expertise of an architect is required to translate those forms into meaningful architecture proposals.Hence, in order to be translated into a functional tool, further exploration is required to more closely understand the differences and changes in the results produced by the methods applied.
However, the introduced pipeline, if integrated into our digital processes, allows for the generation of several different outputs which may not necessarily be immediately related to each other, qualities which align well with lateral thinking approaches.Such an approach, with its numerous variant outputs and experimental nature, might allow the designer to view a problem from a new light or inspire oblique and new ways of thinking.Thus, this new research has the potential of not only generating new mechanisms for latent thinking in design processes but also to allow designers to move away from established ways of thinking and to question the current techniques of designing embedded in our tools that we normally take for granted.

CONC U ION
This paper presents a pipeline for future explorations into designing tools that encourage and highlight experimentation and lateral thinking approaches in computational tools of architecture, qualities that are not as prioritized in our tools today.Specifically, in this paper, we introduce a 4-step AI-driven pipeline that uses the latent space of a machine as a digital environment to design in, and we demonstrate examples of navigating in this space to generate 3D voxel structures.The results of this process aim to act as examples of possible architectural process scenarios and as transitions between current ways to speculative ways of designing.
We hypothesize that such a pipeline, if fully integrated into digital architectural tools, may enable us to incorporate lateral thinking into our current digital processes and pull our architectural tools out of the industrial era thinking.Hence, by reinforcing the uncertain and the unpredictable, the course of the discipline can be read usted with our digital futures.
The paper also brings forth the use of computers as not only tools but also as collaborators in the process of designing and a next step towards human-computer interactions.nlike the digital tools we have today, the pipeline presented, instead of following a direct inputoutput relationship, allows the machine the freedom to work with pixels and contribute its own understanding of the input to generate drawings.This particular space of imprecision and inaccuracy based on machine perception and learnt features is where the potential for imagination and lateral thinking lies.This presents us with opportunities of working with architectural drawings in new and novel ways and could allow the discovery of new spatial and previously "unthinkable" concepts.This also introduces a move from design decisions made by only human logic and biases and allows us to reimagine the ways we design and how we articulate our built environment in new and exciting ways.

R R NC
re ipeline lo chart a Collecte ataset t leGAN training se uence c imensional latent space of a machine electe images an their metho s in the latent space: a Na igation metho : e ual rations Na igation metho : une ual ratios c Na igation metho : mi ing Na igation metho : com ining mi ing an une ual ratios re i ing plans ith ifferent factors re nterpolating from one point ith ifferent coefficients