Context-rich Urban Analysis Using Machine Learning A case study in Pittsburgh, PA

This paper reports on the analytical potential of machine learning methods for urban analysis. It documents a new method for data-driven urban analysis based on diagrammatic images describing each building in a city in relation to its immediate urban context. By statistically analyzing architectural and contextual features in this new dataset, the method can identify clusters of similar urban conditions and produce a detailed picture of a city's morphological structure. Remapping the clusters from data to 2D space, our method enables a new kind of urban plan that displays gradients of urban similarity. Taking Pittsburgh as a case study we demonstrate this method, and propose ``morphological types'' as a new category of urban analysis describing a given city's specific set of distinct morphological conditions. The paper concludes with a discussion of the implications of this method and its limitations, as well as its potentials for architecture, urban studies, and computation.


INTRODUCTION
The relationship between architecture and the city is critical to architectural design.How architecture responds to, or is influenced by, its urban context is a key issue in architectural debates since the modern era (Beigel 2010).In this paper we define context as the physical conditions surrounding a building within a certain range.This context can comprise different kinds of urban artifacts (Rossi 1982) -humanmade objects such as buildings, infrastructures, public space, etc.
Recent research at the intersection of architecture, design, and computation have sought to add nuance and detail to our understanding of urban morphology.Various computational methods in studies of urban morphology have been employed to define and distinguish patterns of a city, such as the statistical model (Colaninno et al. 2011), space syntax (Alexander 1965), convolutional neural networks (Albert et al 2017), among others.These methods suggest a new point of view with respect to urban analysis.However, their datasets include only the feature metrics of individual urban artifacts and hence fail to capture contextual and relational aspects of the city.In this paper we show how including urban contexts has potential for yielding an enriched picture of a city's morphological character and open new avenues for computational urban studies.Complex patterns such as those found in city fabrics can be difficult to apprehend and analyze.The approach to context-rich computational urban analysis we introduce here shows that it is possible to identify "morphological types" based on statistical analyses of urban traits such as building arrangement, density, street profile, hierarchy of circulation, fabric regularity, among others.Our method has the potential to allow researchers to characterize a city's morphological signature, identify its predominant morphological types, and synthesize a data-driven morphological map revealing a city's structure from a new perspective.Rather than replacing, this new perspective has the potential to enrich established methods for urban analysis.The following sections describe our method in technical detail, and illustrate it through a case study in the city of Pittsburgh.We validate our analysis through on-site surveys of boundary conditions between fabrics belonging to different types, and reflect on some limitations and avenues for future work.

DIAGRAMMATIC IMAGE DATASET (DID)
Building data has two different sorts of information: target and context.Shape and height of the building are the geometrical aspects of target information representing the feature of the internal aspect of the object.An arrangement or locational information with other buildings, also geometrical, is the context information representing the relationships.This paper proposes a diagrammatic representation which includes these two types of information and can be employed for an analysis of urban space through machine learning.

Type of Dataset
Before generating a computer-recognizable dataset in architecture and urbanism, the type of dataset must be set and decided.There are two main types of a dataset: raster and vector.Raster or raster graphics refer to images.Otherwise, vector or vector graphics refer to the matrices of the information.
Both types have their own benefits for statistical approaches of urban analysis; in this research we choose raster graphics because it is easier in standardizing the format of information.When the image size is set, an image can represent all contained information by specific modes (e.g., grayscale, RGB, CMYK), as these modes have their own format to represent pixel values in the image.For example, a pixel in RGB mode image can be represented by a threedimensional vector like (0-1, 0-1, 0-1).If image size is 5×5, all pixels can be converted as a vector shaped like (5,5,3), ultimately all different shape information in images can be converted by the same shaped vectors (Fig. 1).A city has a variety of situations, and the situations appear in relation to diverse geometric factors in urban morphology.Therefore, it is more beneficial to create raster images to represent diverse geometric information in a consistent way.

Advantages of Diagrammatic Image
Diagrammatic image is an image containing information as a form of diagrams.It can be used to illustrate relationships in architecture and urbanism, as well as in other fields.Using this data sets, the patterns or features of relationships that appear in diagrams can be investigated, extracted, and analyzed through statistical methods (e.g., deep learning).Currently, there are two ways of creating images of urban information: satellite and map images.However, both methods contain more information than a researcher normally would expect to see.This can result in more noise than relevant information.Compared to these methods, diagrammatic image, as a format of data, offers two advantages: low noise in the image and synthesis of information.
Low Noise in the Image: Diagrammatic images tend to contain less noise than satellite images.A satellite image is an image of the city at a specific point in time.However, satellite images contain elements such as trees, cars, lanes, materials, and shadows as noise.On the other hand, since a diagrammatic image is composed of specific data in the image, it is less likely to contain irrelevant information.
Synthesis of Information: This is the most valuable aspect of diagrammatic images.Unlike other types of urban information images, users can choose the sorts of information to be included in the image.Furthermore, when synthesizing the information into an image, users can control their importance, by distinguishing in the graphical expression method, the means to represent the various information.Specifically, it is possible to create an image reflecting subtle differences by using graphic elements such as color, line thickness, and degree of overlap.
In diagrammatic images, we invented a specific setting for configuring information and distinguishing between the target and context information.A diagrammatic image has target information located at the center of the image, such as target building footprint and target parcel which is the lot boundary of the target building.The context information, such as context buildings, context parcels, and streets with a hierarchy surrounds the target information.With this configuration, the computer can detect morphological relationships between a target and its context information.Fig. 3 shows an example of one such diagrammatic image.It contains information about specific urban patterns surrounding the target building.Target buildings belong to neighborhoods each of which can be identified by a label in the images.As each building can be a target, potentially, a diagrammatic image dataset (DID) has as many diagrammatic images as the given buildings.

Generation of DID for Pittsburgh (DID-PGH)
Investigation of morphological types in Pittsburgh requires a specific DID for Pittsburgh, which we refer to as DID-PGH.The process in creating DID-PGH has two main steps: pre-processing and generation of diagram images.In the pre-processing step, there are three detailed steps: data collection, filtering data, data visualization.
Data collection preprocesses GIS data to include shapes of the urban elements to be contained in the images with their tags.We collect building footprints and heights, parcels geometries, neighborhood boundaries, and streets.For DID-PGH, we mainly use data provided by Allegheny County.
Filtering data is the step of extracting only necessary information from the collected data and matching and synchronizing them to each other.As Allegheny County encapsulate a region larger than the city of Pittsburgh, it is necessary to set a proper range for the data by reducing the data from all regions in Allegheny County to just those in Pittsburgh.The city of Pittsburgh is made up of three large regions (Fig. 4).This research focuses on the region B which contains Downtown.In the sequel, Pittsburgh will refer only to region B of the city.
Data visualization assigns a drawing style to the refined data for use in diagrammatic images.
The process of generating diagrammatic images is done by filtering preprocessed data by the assigned drawing style within the intended scope.After filtering and matching all relevant information within region B, the next step is to set the image range window.This window sets the contextual condition, essentially a range of urban elements related to the target building.The contextual conditions may include a broader architectural relationship, or an immediate relationship such as footprints of neighboring buildings.As this research concerns urban morphology, the range should be set smaller than a city or community scale but larger than adjacent buildings.To get the proper range for the window, we set the average radius of the target building to be the square root of the average area of all building footprints.Assuming that there are three neighboring buildings on either side of the target building, from Fig. 5, half of the window range is set as 98m and therefore, one side of the window is 196m.
Drawing diagrammatic images requires the process to set the diagram drawing style.This style can be customized by user intention.In this research, we filled the building footprints which is centered on the window with a solid color.In particular, the target building has a color from a color gradient according to its height.The color gradient can be segmented as five different colors to categorize the classes of building heights: 1m -12m (max.3 stories), 12m-25m (max.7 stories, low-rise building), 25m-75m (max.20 stories, mid-rise building), 75m-150m (max.35 stories, high-rise building), 150m -(super high-rise building).We only color the target building by its height.For other analyses, it is possible to color all buildings within the window by their heights.Additionally, we can set the graphics style for all the line elements such as parcels and streets, for example, highway as 10 px wide dark blue lines, major roads as 7px wide blue lines, minor roads as 3px wide sky-blue lines, and pedestrian paths as 1px wide light blue lines.Upon assigning the diagram style, geometries appear in the window according to the given style (Fig. 6).In the generation of DID-PGH, the site includes 48,976 buildings and 64,019 parcels.Algorithmbased generative modeling software, Grasshopper®, and customized plug-ins, namely, DID Toolkit, were employed.In the DID Toolkit, we can set two conditions to prevent incomplete information from being included.The first condition is to check whether the target parcel is not empty.If there are no buildings on the target parcel, it is difficult to translate the relationship between the target building and its context conditions.Another condition to check is Under these conditions, the custom component DID generator in Grasshopper looks at 48,913 buildings and generates 45,852 diagrammatic images with 3,061 images not satisfying the given conditions.The label file has 45,852 integers matching the number of diagrammatic images.On an 'Intel(R) Core (TM) i7-8700k @ 3.70GHz' processor with 64GB memory, it took 49.72 hours to check 48,913 buildings to create the 45,852 diagrammatic images.

MORPHOLOGICAL TYPES Clustering Based on Contextual Similarity
In statistics, machine learning, and information theory, dimensionality or dimension reduction is the process of reducing the number of random variables under consideration by obtaining a set of principal variables (Roweis and Saul 2000).This section shows how these statistical methods can detect similarity of architectural context conditions of a building in a di-agrammatic image through their visual information.Dimensionality reduction is employed to extract the main features of the architectural context conditions in an image.
We used the t-SNE (t-distributed Stochastic Neighbor Embedding) algorithm to reduce dimensionality and keep principal variables of DID-PGH.t-SNE is a tool to visualize high-dimensional data.It converts similarities between data points to joint probabilities and tries to minimize the Kullback-Leibler divergence (Kullback and Leibler 1951) between the joint probabilities of the low-dimensional embedding and the high-dimensional data (Sci-kit Learn).In these experiments, the t-SNE algorithm is established and visualized in Tensorboard® by using an embedding projector.The hyperparameter settings are as follows: perplexity 35, learning rate 100, iterations 1500.A data point in the reduced dimension space means the building, the location of the point means its characteristics of the building footprint and architectural context conditions, and the distance among the points means the degree of the similarity.The shorter the distance, the more similar the context conditions.The left image in Fig. 8 shows the results of visualization of DID-PGH through t-SNE based on the similarity of architectural context conditions by neighborhoods.
Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm is used to find core samples of high density from the distributed points (Sci-kit Learn).After applying DBSCAN to the results of the t-SNE visualization, the data points distribution model illustrates similarity of the contexts by the distance between data points.The hyperparameter settings of DBSCAN are as follows: Epsilon 1000 (10 × 100), the minimum number of samples 15, leaf size 30.The data on the normalized data space from 0 to 1 is imported to Rhinoceros® using Grasshopper.The data space is scaled by a factor of 100 for improved visualization.The Scikit-learn's DBSCAN was invoked within Grasshopper.The value of epsilon was scaled accordingly.According to the results of the DBSCAN (Right, Fig. 8), in data space, Pittsburgh has 21 different distinct clusters that share morphologically similar architectural contexts.

Data-driven map of Morphological Types
From the clustering results, two types of urban fabric map can be derived: a data-driven map of morphological types (DMT) by buildings and borders.We remap the clustering results on the real-world map by coloring each building with the clustering labels from the data space.By coloring the footprints with different colors, a map containing the building footprints by the clusters from the reduced dimension space using t-SNE and DBSCAN can be generated (Left, Fig. 9).
After marking the buildings in the map with their cluster color, boundaries can be generated by a KNN (k-nearest Neighbors) algorithm.First, we set the data point to be used as the data KNN by extract-ing the center point of each building footprint.Additionally, we create the label file from the clustering labels for KNN.The hyperparameter settings of KNN are as follows: algorithm 'Auto' , leaf size 30, metric 'minkowski' , metric parameters None, number of jobs '1' , number of neighbors 3, power parameter for the Minkowski metric 2, weights 'uniform' .Original boundaries of the KNN were fit to a rectangular window.We trimmed out the original boundaries with the outline of Pittsburgh to create the DMT for boundaries (Right, Fig. 9).

Morphological Analysis by Clusters
With DMT, typical context conditions of new urban fabrics are sampled to examine the properties of urban clusters composed of 21 clusters.3D models of typical context conditions of 21 different morphological types show their spatial features and information.We sample the center of the largest type boundaries for selecting the sites of 3D models.In order to investigate the distinctive features of each morphological type, we used spatial quality descriptors and several statistical values: occupancy rate (%), occupancy rank, average height (m), average area (m2), average density, average building occupancy, and average stories.
'Occupancy Rate' is a relative measurement of how much a cluster covers the city area.'Occupancy Rank' is the rank of the occupancy rate.'Average Height' is the mean of the building heights in a cluster.'Average Area' is the mean of the building areas in a cluster.'Average Density' is the mean of the building densities in a cluster.Density is the floor area of the building multiplied by the total height divided by the floor height (3m).'Average Building Occupancy' is the average of the building occupancy which is the ratio between the building floor and parcel area, 'Average Stories' is the average height divided by the floor height (3m).
We introduce some characteristic types in this paper.
More information about each type can be found on the website (http:jinmorhee.net/dufm.html).In the sequel we The most popular urban type is 'Urban Type 14' , which presents places where a certain number of empty parcels exists in the neighborhood and has no distinctive shape of blocks.The next popular type is 'Urban Type 1' , which is street-centered and has orthogonal intersecting spaces with different sized urban blocks.This is mainly distributed in Lawrenceville, Bloomfield, Shadyside, and Oakland.The vertically distinct urban type is 'Urban Type 8' , which consists of typical blocks with grid systems.This is mainly distributed in Downtown and the Strip District.Urban types with a relatively large building area are 'Urban Type 13' and 'Urban Type 15' .These are mainly distributed in Garfield, North Squirrel Hill, Green Field, and Swisshelm Park.Urban Type 6 has the highest level of land occupation.It can be interpreted that this fabric has the least outside space.When 'Average Area' , 'Average Density' , and 'Average Building Occupancy' are interpreted together, it can be seen that 'Urban Type 13' and 'Urban Type 21' have the highest density of buildings with the high level of FAR (Floor-Area Ratio).

Conflicts among Morphological types
Based on the DMT, examining conflicts among the urban morphological types is valid and applicable to urban form analyses.Investigating conflicts of urban morphological types begins with selecting the sites.We focus on two locales in Pittsburgh: Shadyside, where large fabrics collide, and Squirrel Hill, where small fabrics collide.In these two locales, the main conflict points are set around the streets.We record not only the situations of conflicts among urban fabrics by tracing photographs, but also the inside situations of the types, to see how the typical conditions form the actual urban space.In this paper, we look at one example of these conflicts.More information about the conflicts can be found on the website (http:jinmorhee.net/dufm.html)(Fig. 11).
There is a conflict in Shadyside, at the intersection of Pembroke Place and St. James Street (marked as 'a07' on Fig. 11).This point of conflict shows the borders between Shadyside's expensive price area and average price area.They have different architecture types: mansion style house (Urban Type 19) and compact collective house (Urban Type 1).It can be seen that even within the same neighborhood there can be different architectural and urban environments.This conflict is clearly described in Fig. 12 facing the north and south at Pembroke Place and St. James Street.The east side of the street is the vernacular of common collective housing in Shady Side.This type of housing forms the urban space in a way that similar size buildings are tightly lined up.The west side of the street has the buildings largely occupying the ground located at a distance from each other, called mansion-styled houses as the type of independent residence with individuality.

CONCLUSION
This paper proposes a new way to analyze a city considering urban contexts, rather than individually analyzing the urban elements.We created data including the morphological characteristics of the buildings and their surrounding urban spaces.By classifying the data, statistical techniques identified the morphological features and types of a city.
The map of morphological types generated using our method innovatively considers contextual information for 45,852 buildings codified in a novel type of urban dataset (DID).Using machine learning techniques, our method classifies and clusters buildings according to their relationships with their surroundings.From this clustering we identify morphological types as a new category in computational urban studies.Morphological types are defined as high-level representations of frequent urban conditions identified computationally in an urban contextual dataset.As discussed above, these morphological types form patterns which paint a morphological picture of a city.This morphological map reveals boundary conditions, gradients, and map the prevalent kinds of urban fabrics comprising a city.

Figure 13 The Concept of Reverse Analysis for City
The value of the urban analysis technique using DID and machine learning is not limited to simply mapping urban fabrics in new ways.What is more remarkable is in the approach of this method to help us read and interpret the city in new ways (Fig. 13), and synthesize legible insights from large amounts of information.A city can be seen as a data space where this granular and numerous information is well-suited and intertwined.In the existing GIS, the city itself was regarded as a collection of information, rather than as a complex data space.This recognition does not interpret the data from a computational point of view but still allows the data to be processed from a human perspective.By segmenting into states, cities, neighborhoods, streets, and parcels, people have classified, categorized, and understood the complex data of cities.This is an urban interpretation derived from approaches based on segmentation.
If people have a tool that allows them to observe a myriad of infinite pieces of information one by one, they can look at the various information in the city and identify certain rules and patterns.They can find and differentiate the overall patterns in a dataset by integrating individual data instead of segmenting it.The possibilities of AI technologies in architectural design is amplified when thoroughly dealing with a data space called a city with complex and innumerable relationships.If people have a new method of integral logics to calculate the overall shapes of urban data rather than segmenting them, it will be another way to interpret the city analyzing the implicit and explicit phenomenon more flexibly and efficiently.
Avenues for future work include, first, a comparative approach which applies the method presented here to the analysis of different cities.The spatial and morphological identities of different cities, their morphological signature and types, can thus be studied in ways that extend other forms of urban analysis.Second, the DID dataset can be enriched with information related to non-physical urban traits such as historical information, census data, etc.This may offer new ways of investigating the relationship between socio-economic phenomena and urban form.Third, a limitation of the current dataset is its lack of three-dimensional information -such as terrain and building mass.By designing richer representations, new questions and new levels of accuracy may be asked of this approach.Fourth, there is a great potential for our analytical method to support generative urban design and planning methods.Finally, this research opens questions about the practices of urban data collection and analysis, and about the nature of the data itself.As the project evolves, more research is needed to develop and document a more nuanced and situated understanding of the data used, and the materiality and historicity embedded in its very design.
Figure 1 Example of Vectorized Value from Raster Images Figure 2 Three Types of Urban Information Image: Satellite Images, Map Images, and Diagrammatic Images Figure 3 An Example of Diagrammatic Image Figure 4 Regions of Pittsburgh Figure 7 Part of DID-PGH, Arranged by Neighbor Density Figure 8 t-SNE Visualization of DID-PGH Based on the Similarity of Architectural Context Conditions by Neighborhoods (Left) and Results of the Clustering through DBSCAN (Right) Figure 9 Data-driven Map of Morphological Types by Building Footprints (Left) and Boundaries (Right) Figure 11 Site Survey, Places Where City Fabrics Conflict Figure 12 Pembroke Pl. and St. James St., Facing South (Left) and North (Right)