Spatiotemporal Occupancy for Building Analytics

Numerous studies on Space Syntax and Evidence-based Design explored occupancy and movements in the built environment using traditional methods for behavior mapping, such as observation and surveys. This approach, however, has majorly focused on studying such behaviors as aggregated results -totals or averagesto corroborate the idea that people's interactions are outcomes of the influence of space. The research presented in this paper focuses on capturing human occupancy with a high spatiotemporal data resolution of 1 sq.ft per second (0.1 sq.mt./s). This research adapts computer vision to obtain large occupancy datasets in a hospitalization setting for one week, providing opportunities to explore correlations among spatial configurations, architectural programs, organizational activities planned and unplanned, and time. The vision is to develop new analytics for building occupancy dynamics, with the purpose of endorsing the integration of a temporal dimension into architectural research. This study introduces the ``Isovist-minute''; a metric that captures the relationship between space and occupancy, towards a point of interest, in a dynamic sequence.

tions are qualified by spaces" (Tschumi 1996, p.130). This paper is situated in the interplay of the aforementioned spatial and non-spatial dimensions of a building setting, and proposes a set of techniques to capture and analyze one aspect of human behavior: occupancy, with the purpose of demonstrating the value of behavioral related metrics.
For several decades, numerous studies on the areas of Space Syntax and Evidence-based Design (EBD) in Architecture have focused on movements and occupancy in the built environment (Alexander, 1977;Hillier, 1989;and Zimring, 2004 among others). This approach has majorly focused on studying such behaviors as aggregated results, counting totals or averages to corroborate the idea that people's interactions are outcomes of the influence of space (Bafna, 2003), but disregarding the temporal character of building occupancy, in particular with a high spatial and temporal resolution. Specific questions addressed in the aforementioned studies, such as how movements are displayed in specific spatial settings; how occupancy is distributed in specific spatial configurations; or how spatial characteristics correlate with such behaviors, may be attributed to the challenges faced when trying to collect occupancy data of high temporal-and-spatial resolution. It is common to see the use of traditional methods, such as observation and manual mapping, which is very accessible since they rely on human abilities. However, other methods recently matured into commercially available tools, such as Computer Vision or the use of Wearables, offer a remarkable opportunity for collecting occupancy data of high spatial and temporal resolution.
The importance of obtaining high spatial-andtemporal resolution occupancy data lays on allowing the study of building occupancy dynamics. "Occupancy and movement data are crucial to deepen the understanding of the build environment performance" (Tome and Heitor, 2015).

Figure 1
Intensity of research distribution regarding spatial and temporal resolution. Dark gray indicates more research (Example of a Research Question: "What is the role of spatial layout in shaping the way in which visitors explore (the space)...?"); Light gray indicates less research (RQ: "How the configurative properties of leaning settings influence the users' experience of the space?" and, "What is the length of stay of a patient in an exam room?"); Finally, color indicates this research targeted position within the existing research context.
The improved dataset allows advanced methods for data analysis to promote new architectural research, such as exploring questions that fit the structures:1) "what is the appropriate data resolution to answer a specific question in a particular scenario?"; and 2) "what is the best approach to capture the data to achieve such resolution?" (Liu, 2007). Also, other questions emerge, such as what are the aspects to consider in defining the scenario to allow for plausible generalization? And, what are the suitable models to describe occupancy patterns in buildings? Each of the above questions poses significant challenges and research opportunities.
This research proposes to automatically collect high resolution spatiotemporal occupancy data in the built environment (1 sq.ft. per second) by adopting and adapting existing systems and computer vision methods. Although research have been conducted on developing wearable systems, triangulating data using WiFi networks and embedding sensors in the environment. The innovation of this research focuses on the resolution and the large size of the datasets obtained, which unlock a wide range of research questions, which have not been explored yet. Those questions fit on previous frameworks, but they are fine-grain tuned, providing opportunities to explore correlations among spatial configurations, architectural programs, organizational activities planned and unplanned, and time. The dataset is explained as outcomes of a statistical model, seek-ing to demonstrate that the characterization of occupancy depends on the influence of spatial parameters as well as architectural program and fine-grain activity factors, distributed in time.

OCCUPANCY IN ARCHITECTURE
Most of the studies have focused on the unidirectional influence of layout on human behavior, analyzing movements, flows, occupancy, and interaction among individuals in different layout configurations, as previously reviewed. These studies have answered questions such as how open-layouts influence patterns of exploration (Peponis, C.Dalton, Wineman, Dalton, 2004) or the role of the physical environment in a hospital setting (Zimring, Joseph and Choudhary, 2004), to name a few. Some findings reported that in complex layouts, such as museums, the spatial configuration restricts human movements and directs their flow and viewing patterns (Bafna, 2003). In re-configurable layouts, such as offices, "spatial configuration directs movements and influences the field of vision, directly affecting the co-presence of people and their face-to-face interaction. " (Gomez et al. 2012 a and b;Choi, 1999). Such findings provide evidence that layouts, in fact, influence behavior and movements; however, most of the conclusions reached by these studies have been based on aggregated behavioral data correlated with geometrically derived attributes of space only. The range of such research questions has not broadened since data spatial and temporal resolution cannot increase as long as the data collection methods require human intervention and interpretation. Despite these observations, such traditional methods of data collection are very popular and accessible.
The characteristics of behavioral data collected through these classical methods lead the research questions toward general descriptions of behavioral data patterns, presenting three main issues that are central to this thesis. First, the investment of resources and the continuous presence of observers. Second, the variables collected usually refer to humans' position in space, their role, and particular event possible to be captured within a human reaction timeframe, which is usually in the range of a minute. And third, the spatial and temporal resolution of the data obtained is limited to human capacity to observe and annotate. To overcome these limitations, research in computer science has developed a series of positioning systems and algorithms. However, very little architecture research has incorporated them yet. Since 2009, very few studies in architecture have adopted new technology for collecting behavioral data. Some attempts included the use of radio-frequency identification (RFID) proximitytracking sensors , followed by scene analysis systems based on video capture (Gomez et al., 2012(a and b); Tomé and Heitor, 2012, 2013 and 2015 a and b), and proximity and triangulation sensors (Erickson, Lin, Kamthe, Brahme, Surana, Cerpa, Sohn, and Narayanan, 2009). Two major differences between these studies were the focus on occupancy versus movements and the time aggregation range, with the first study having a time resolution of one second and the second having a time resolution of one hour. Other architectural-related Computer Science studies with a focus on technology development are also crucial to this research, i.e., "Vis-A-Viz" (Romero, 2008), "The History of Living Spaces" (Ivanov, 2007), and (WeWorkBIM, 2015). All the studies utilized information visualization techniques, including time as a variable for human occupancy and movement analysis.
Within the context of existing technologies, a remarkable opportunity for collecting data of high spatial and temporal resolution arises, allowing enhanced methods of behavioral data collection and data processing to promote new research analyses, such as the influence of organizational activitiesboth scheduled and unscheduled in time-on occupancy patterns.
An exhaustive review of each system and technology was done based on previous research and the products' online reviews (Gomez Z., P., 2017) for selecting the system. This review included ten specific technical and social criteria: accuracy, precision, range, dimensions, temporal scale, robustness, cost, privacy, passiveness and pervasiveness. Each of the analyzed systems provides occupancy data of different characteristics in terms of the criteria aforementioned, establishing an interdependency between a specific architectural research question and the positioning technology that provides the appropriate data to answer such a question. Hence, the problem becomes a multi-dependent problem, presenting a platform that, based on the review of positioning technologies, helps to technically select the appropriate methodology for high-resolution spatiotemporal occupancy data collection and analyses. It also supports the co-evaluative process of the research question formulation in correlation with both the positioning tools and the dimensions of the built environment possible to capture. This platform indicates the influence of one research variable over the next one, in a chain, providing a matrix of research possibilities (Figure2).
Once one "best-fitting" system was selected for this study, the potential research question is narrowed down to a specific building program, social context, scenarios and data resolution and accuracy. In this particular proof-of-concept study, a hospital in Chile was selected, the scenarios were determined by the organizational schedule, and the data resolution was one square feet per second or higher. To achieve such resolution, Scene Analyses using Computer Vision was the selected method for positioning information. Specifically, the first challenge is to define the required data characteristics, consequently, the appropriate system that would allow capturing such data to answer the specific question of interest that belongs to the specific scenario. Additionally, a list of parameters that describe the diverse environmental conditions for determining accuracy in the automatic detection is proposed, shifting the focus to answering research questions that are sensible to the scenario's significance, and at the same time providing insights for meaningful detection in context of Computer Vision research. This research is presented in detail in a separate publication (Gomez, P., 2017).

SCENE ANALYSIS
Scene Analysis was tested in a real scenario, collecting data for one week on a Hospitalization building in Chile, as a proof-of-concept. The aim of the study was to investigate practical and methodological issues involved in using scene analysis to collect occupancy data at high spatial and temporal resolutions, and to demonstrate how new measures of occupancy rates can be developed in order to understand the relationship between spatial configuration and behavior.

A Hospitalization Setting
The key aspects involved in selecting the scenario are the architecture program type and specific areas of interest. The selection of a strong program -a hospital -helped to restrict the programming variables. Moreover, the corridors of the hospital, -which share an almost identical layout configuration among floors, have no specific program or activity assigned to them other than their role as connectors; therefore, their description is, by sections, linked to the adjacent spaces. These constraints frame the contributions of scene analysis for occupancy detection in a restricted spatial context, assigning most of the accountability for the outcomes to activities scheduling and time, building the argument from a technical as well as theoretical perspective. Also, a strong healthcare program was selected based on the evidence produced by the EBD and spatial analyses fields of study, and the interest that healthcare has received because of its organizational complexity and its effects on people's well-being.
This research addresses the proof-of-concept study by processing one week of surveillance videos over twelve corridors of the hospital in Chile. The resulting occupancy was captured at a spatiotemporal resolution of one sq.ft (0.1 sq.mt.) per second using Scene Analyses methods based on Computer Vision. Scene Analysis allows for the collection of high spatial-and-temporal occupancy data using existing surveillance recordings from a hospital, avoiding interference with the social aspects of pervasiveness of technology into daily routines. Afterwards, computational methods adapted from Computer Vision rely on adapting pre-built algorithms for automatically recognizing occupancy. These algorithms can be trained and have a certain accuracy range influenced by environmental as well as occupancy parameters, which can be represented by a multiple regression model.

Scene Analysis Process
This section describes the general approach to obtain the spatial and temporal data needed for further spatiotemporal analytics. The details on data collection and processing, and accuracy and precision corrections are described in Spatiotemporal Occupancy in Building Settings (Gomez, P. 2017). After obtaining the video files from the server, computer vision methods that rely on pre-built algorithms for automatically recognizing people, were adapted to detect occupancy and its accuracy (Figure 3). The actual data collection provides the empirical perspective, introducing specific methods of scene analysis including video acquisition, video processing, adaptation of computer vision detection algorithms, as well as the practical and technical challenges faced in obtaining the required high-resolution occupancy data results.
The data accuracy range influenced by environmental as well as occupancy factors (Romero, 2008;Tome, 2015), can be represented by a regression model. The hypothesis is that the statistical model will help us understand the spatiotemporal occupancy characterization, independently of the influence of the data source accuracy. The last stage presents the computation of the accuracy of the oc-cupancy recognition algorithm by explaining the environmental and occupancy factors that influenced the errors obtained. Once the occupancy data are discerned, the goal is to measure the relationship between space and behavior in a specific scenario, delineating and developing a new key occupancy metric: Isovist-minute, which is defined as the relationship between real and probable occupancy and their temporal visual fields towards a target. All the methods proposed along this research are general enough to be applied to any specific built environment; However, this research focuses on occupancy of individuals in hospitals corridors, as they are a weak-program spaces into a strong building program. The corridors are characterized by its layout complexity, high spatial segregation, strong control over spatial divisions, strong control over the use of spaces and activities assigned to spaces, and strong control over inhabitants' routines and visitors' schedules (Koch and Steen, 2012).
Nevertheless, the main focus of this research is not the adoption of the scene analysis using surveillance systems, but the collection of high spatial-andtemporal occupancy data avoiding the interference with the social aspects, such as the pervasiveness of technology into daily routines, which in turn directly compromises the legitimacy of the data collection Koch, 2012), and the creation of new architectural metrics to measure building organizational performance.

SPATIOTEMPORAL OCCUPANCY MODELS
A spatiotemporal occupancy model refers to the structure and presentation of the collected occupancy data, organized and displayed to support its exploration. Its purpose is to facilitate the identification of occupancy patterns to explain the theoretical influence of "organizationally scheduled activities" on people's behavior, with the main goal of characterizing the occupancy patterns of a particular building.

Figure 4
It shows the 2D occupancy gird size and grid indices (row,col), staring at the upper left corner (0,0). The spatiotemporal occupancy data is stored as a CSV file because it provides a manageable file size to accommodate the amount of data stored in this case. Each occupancy record consists of three fields in the automatic data collection (x, y, and time) and ten fields in the manual data collection besides x, y, and time. These ten fields include lighting type, lighting quality, and camera condition as environmental conditions, and role, activity, body posture, activity type, activity frequency, occlusion, and cloth color as occupancy conditions. Each CSV file contains occupancy data by hour and by corridor. The mechanisms for exploration are the accumulation of a set of hours or the comparison among a set of files. The graphic display of the data is structured as x and y on a two-dimensional (2D) occupancy grid and with a z-axis temporal dimension to construct a three-dimensional (3D) "occupancy cube, " referencing Mario Romero's work on 'activity cube ' (2008).
The occupancy grid in this research is a finegrained 2D map composed of a collection of cells representing the continuous space of possible locations in the scenario, reducing the spatial complexity (Figure 4). Additionally, a set of 2D occupancy grids are sequentially aggregated together by time instances in a Z-axis to create a spatiotemporal 3D occupancy cube. Both the 2D occupancy grid and the 3D occupancy cube are based on a grid-based approach, which is computationally easy to build and represent, providing independent locations. The first gridbased map was originally proposed by Elfes (1987) and Moravec (1988), who assigned an occupancy binary value to each cell to determine its occupancy. Occupancy grids have also been defined in the area of probabilistic robotics, the goal of which is to "estimate the posterior probability over maps given the data" (Thrun, Burgard and Fox, 2005).
The resulting aggregated 2D occupancy grid stores a set of the binary occupancy values (0 and 1) assigned to each cell, resulting in a 2D binary occupancy grid or a weighted occupancy grid, which reflects the aggregated results of the temporal occupancy grid. The values for each cell refer to the total amount of time the cell was occupied during a period of time, representing the percentage of the total time the cell was occupied. Also, each occupancy cell imports the occupancy probability values calculated from the regression models. The fundamental differences between occupancy probability and weighted occupancy are that the occupancy probability values are calculated from the accuracy results and are expressed as percentages, while the weighted occupancy values are calculated as the sum of the occupancy temporal duration during a determined period of time. These results are expressed first as integers, representing the time units that the cell was occupied, and later, normalized as percentages.

ANALYZING SPATIOTEMPORAL OCCU-PANCY
The purpose of analyzing spatiotemporal occupancy is to understand the fundamental characteristics of occupancy in terms of spatial and temporal distribution in different scenarios, with the main goal of determining key indicators for a building's data-driven occupancy performance. The spatiotemporal analyses consist of the following four stages: 1) a visual exploration of the data by importing subsets of activity types to provide insight about the occupancy distribution of sample scenarios; 2) the quantitative analysis of occupancy distribution to compare the scenarios, determining their statistically significant differences; 3) the definition of spatiotemporal parameters based on the visual understanding of the occupancy data subsets; and 4) the definition of spatialbehavioral metrics based on the spatiotemporal parameters to specify precise built environment performance indicators.

Figure 6
Comparison of 4 sample scenarios. Columns from left to right indicate the corridor organizational unit: General Hospitalization and ICU; and a sample day: Wednesday. Rows indicate the two one-hour samples, two times a day: Medical Rounds at 8 am and Visiting Hours at 4pm.
The goal of comparing samples scenarios was to unveil the differences in occupancy distribution based on the type of activity. The comparison occurred be-tween several of the scenarios' occupancy patterns under the same spatial conditions but different programming and scheduling conditions. The nine sample scenarios selected to show these differences were medical rounds, visiting hours, and no scheduled activities in two corridors and on different days of the week. The three types of activities occurred at 8 a.m., 4 p.m., and midnight respectively, in all corridors. The corridors selected had two different organizational programs: the 7th floor, which contained private rooms, and the 3rd floor, which contained the ICU. The two days of the week selected were Wednesday and Saturday (Figure 6).

THE ISOVIST-MINUTE: A NEW SPATIOTEM-PORAL OCCUPANCY METRIC
Few previous studies have proposed Isovist field metrics that originate from patient beds. The targeted visibility study (Lu, 2011), discussed above, counted the heads of patient beds that were in sight of each position from the corridor, assigning this characteristic to the space. Another study by Osman (2016), defined Isovist-connectivity as the properties of each patient bed's Isovist fields, such as its internal connectivity, determining their influence on patient's outcomes, such as mortality rates. Both metrics were founded on three geometrical -and staticparameters of space: visual fields, positions of the beds, and connectivity. These studies, however, did not include occupancy data; which left no information about actual surveillance. Therefore, importing actual spatiotemporal occupancy data will determine the transformation of static-geometric metrics into a parametric spatiotemporal occupancy-related metric of actual surveillance, which would vary depending on the values of the time-and positionparameters. Consequently, this research proposes a new spatiotemporal occupancy-related metric for measuring the performance of the occupied space: the Isovist-minute.
This specific study proposed the Isovist-minute; metric that captures the relationship between space and occupancy, and the projected directional Isovist Figure 7 Visual description of an Isovist-minute computation: At the top of the image is the hospitalization layout, represented by spaces. The pink weighed grid at the center represents the occupancy by cell, during a period of time. The gray Isovists represents the fields of view from the patient's beds 16-c (corridor) and 6-w (window). At the bottom of the image, the black arrow indicates the visual direction from the nurse station to the patient's bed, blocked by the bathroom wall. The heat-map indicates the Isovist-minute of patient's bed number 16-c. (Benedikt, 1979;Turner, 2001) towards a point of interest, in a dynamic sequence. An Isovist-minute of a target is defined as the visibility of a specific target during a determined time frame. In a hospital, for instance, it can be computed for a patient's head, where it measures the amount of visual surveillance of a patient's bed from the corresponding target's Isovist areas in the corridor (see figure 7). The Isovistminute is calculated based on the actual occupancy data collected automatically and corrected statistically as described previously, measuring the real and probable surveillance of a hospitalized patient. The Isovist-minute of the patient, or "target, " measures the time that the head of a patient's bed is within any observer's sight or probable sight. This probability takes into account the statistical accuracy and precision corrections presented, as well as the probable direction of the head of the occupant. The Isovist-minute output is a value that represents the total time during which the target was observed or potentially observed.
Computationally, the Isovist-minute is calculated as a sub-set of a 3D occupancy array of cells containing positioning and time information (x,y,t). This approach is based on both the ease of the computational resources required and the MATLAB language. The calculation of an Isovist-minute requires a description of the patient's fields of view and of the spatiotemporal information occurring inside those fields. The first step is to describe the patient's visual fields -or Isovist-fields -from the head of each patient's bed, and the next step is to define the subset of occupancy cells that belong to each particular Isovist-field. The patient's Isovist-field is defined as the maximum field of vision generated from the upper-third of the patient's bed ( Figure 6). Some cells, as expected, belong to several Isovist-fields (Figure 8). In this research, every hospitalization wing contains 22 rooms ( Figure 9) with one or two beds per room positioned either at the center of the wall (m), the corridor (c) or the window (w), thereby creating several potential Isovist areas. Once the Isovist-fields are created from the head of each patient's bed, the array of cells that belong to that Isovist is stored as an independent CSV file containing an array of cells (x,y), named as "Floor level, " "Room number, " and "Position of the bed".

CONCLUSION
The vision for this research is to develop and recommend new analytics for building occupancy dynamics. The outcomes of this research include a framework that formulates the relationships among the scenario aspects, generalizing the methodology for collecting and analyzing high-resolution spatiotemporal occupancy data. It includes the guidelines for the diversification of scenario conditions focus on answering a variety of research questions that are weighted by the scenario's significance; a proposal for understanding occupancy patterns by constructing the appropriate statistical models to compare and contrast the data; and the emergence of new metrics that respond to the performance of a particular scenario. This paper focuses on spatiotemporal occupancy data collection and analysis methods, to endorse the integration of a temporal dimension into architectural research.