Design Support to steer Creative Wicked Problem Solving Processes with Knowledge Management and Artificial Intelligence

Models, as a representation means for objects, are identified by three main features (according to Stachowiak ⁴):

Models fulfill a replacement function (a) for certain identifiable and/or acting model-related subjects (b) within certain time intervals (c) restricted to certain mental or actual operations.

Thus, models are not only models of something - they are also models of someone. They fulfil functions within a defined period of time and ultimately serve a specific purpose. In order to understand and grasp the complex real world, an abstract image (model) of it is created. The creation of the model pursues a model purpose. For example, models can be used to better understand the real world or a design intention. The purpose of the model is determined by the actors.

In the 1980s, the artificial intelligence (AI) approaches, such as case-based reasoning (CBR), influenced the design of what was to be led to AI research in architecture, the CBD. The vision is a computer system that, based on human thinking, processes experience in cases, and then draws conclusions independently and produces drafts in this way. Since the 2000s, research has focused on supportive approaches of CBD, since, among other things, the formalization of the cases is unsolved in the building industry.

The related work has a decades-long history with a multitude of projects that were initialized during this time. The conceptualization phase of design (for example, in architectural design) is one of the central aspects of CBD. Overviews of the developed methods and applications for architectural design, are, for example, Heylighen ⁵ and Richter ⁶. An essential work of Richter ⁷ summarizes the CBD-related research by providing in-depth descriptions and analysis of the developed approaches.

More or less in the same time period, object-oriented approaches were transferred to the building industry for the formalisation of building information called "Building Information Modeling" (BIM), but the approaches (CBD and BIM) were not successfully linked. Software platform dependent and independent data models were developed, e.g. the so-called digital semantic building models, such as the Industry Foundation Classes (IFC). Despite numerous technological developments, the design process of architecture in the (preliminary) design or concept phase is today insufficiently supported by computers. It is mainly focused on digital representation methods, which are mostly used as transmissions of classic analog design tools and work processes that can be implemented on the computer. Traditional graphic and pictorial working methods, such as freehand drawings or building plans as a reference, are only inadequately supported and continue to remain predominantly traditional.

In the following sections, we present models to formalise information, interaction, data, and retrieval to support building designers in the early design stages.

In the early stages of design, vague, blurred, and incomplete information is usually available in the form of fragments from the requirements of the client (idea or space program). The mostly contradictory requirements are weighed against each other, design decisions taken and a conclusive design is described textually and graphically. The information from the experience of the designer influence the design and the search strategies, since the corresponding technical terms used for the keyword-based search must be known. Therefore, knowledge of the domain is necessary, however, the understanding of the technical terms in the domain can also vary.

Based on the methodological considerations, criteria and characteristics of floor plans are formally recorded and semantic fingerprints (see Figure 1), such as accessibility etc., are developed. Current classifications on the Internet (e.g. detail.de) and classical media (e.g. floor plan atlas of housing construction) are examined as well as bibliographic classification systems and systematics. A further aspect is the analysis of the IFC specification with regard to the possibilities of extracting implicit and explicit data and in particular topological relationships.

Figure 1.Semantic fingerprints.

working out the criteria of the use case,

analysis of the current practice in the field of application

analysis of technologies in the field of application

definition of semantic fingerprints

conception of the knowledge-based thinking tool and

evaluation of the chosen approaches.

Based on the IFC specification 2x3 ⁸, topological and functional properties of floor plans are recorded in catalogues of criteria and 12 fingerprints are derived. With regard to the transferability of the methodology, fingerprints are developed on the basis of analyses and simulations, such as fields of vision and visual graphs.

Starting from the objective, work processes in early design phases and keyword-based research methods are analysed, including the working methods and tools used. In particular, the formulation of search queries and the feedback of the results to the design process are at the centre of the analysis and are considered in a differentiated way for the respective sections of the design process. The paradigm of case-based reasoning is considered, with a focus on the "retrieve" step from the CBR cycle of Aamodt and Plaza ⁹. As in early architectural design phases the focus is on the formulation of the spatial situation with sketchy, graphic and pictorial representations, methods and tools of the formulation and representation of the visual representation (such as requirement matrices, bubble diagrams, sketches, and dimensioned drawings), computer-aided applications are examined with regard to the criteria developed (such as accuracy, detailing and perfection of the representation).

Drawings are an established method for externalizing, representing, and communicating design ideas for spatial arrangements. Research strategies with floor plan-like abstracted representations, such as schemata for the formulation of search queries, are therefore examined. References are analyzed, characteristic drawing and spatial features are derived. For this purpose, computer-based methods for analysis, data modelling, query, similarity determination, and representation of references are investigated. For this purpose, paradigms for purpose-oriented human-computer interaction, especially sketch-like and graphical interaction forms, are investigated and evaluation criteria for the user-centered design of software interfaces are established.

With the implemented interaction prototypes, the derived criteria for the formulation of the search query are evaluated. In addition to the three focal points (text-based input, sketch-based input, and geometric modelling), various hardware environments are included. In a comparative user study, it is examined whether the operating models (see Figure 2 and Figure 3) of the MetisWebUI and Touchtect prototypes ¹⁰ by the target group architects, i.e. whether they are able to express their design concepts with the help of the prototypes. In addition, interaction models are developed in the sense of explanation-aware computing, in which the user is presented with an allocation of rooms entered and rooms found in the search result.

Figure 2.Metis Web UI - software prototype to draw a query with a mouse.

Figure 3.Touchtect - software prototype to perform freehand drawings.

The interaction models developed allow each room to be described with the help of a series of optional properties (geometry, function, presence of natural light, etc.), thereby iteratively refining the space concept developed. By the optionality of these characteristics it is possible to describe rooms incompletely, which increases the freedom of the user. The abstractions of the fingerprints overcome the inaccuracies in the floor plan that occur due to the optionality of the entered floor plan. The user acceptance of the operating models of the Metis-WebUI and of touchtect is determined on the basis of questionnaires, whereby it is shown that all test persons are able to express their design concepts to a large extent with the help of the prototypes mentioned ¹¹. Furthermore, no significant preference for an interaction form / input metaphor and representation form could be determined, so that for the interaction model a flexible input and representation as well as a change and/or compression of the information must be made possible depending on the device.

Investigations are carried out to derive graphs (fingerprints) and to build a graph database (see Figure 4) from IFC files (Industry Foundation Classes). Floor plans are described as undirected graphs, where rooms are mapped as nodes. The relationships between rooms, e.g. direct connections through doors or adjoining walls, are represented by edges. To support the extraction of data from BIM models and the formalization as fingerprints, IFC specifications as well as the description of spatial-topological and geometric information are analyzed. The identified information is taken from digital building models, whereby the Industry Foundation Classes (IFC) represent the underlying standard. Relevant features are extracted via serialization of IFC models. Explicit fingerprints, e.g. number of rooms or accessibility graphs, can be derived directly from the interpretation of the IFC. Analysis methods for IFC are developed and implemented for the automatic extraction of explicit information. Implicit -spatial-topological- fingerprints such as distances between rooms or visibility relationships are not explicitly mapped in the IFC and must be extracted algorithmically.

Figure 4.Spatial configurations abstract in graph database neo4j

In a first step, existing graph databases are examined. The Neo4j technology is favored as the basic technology according to the requirements (graph-based representation of the floor plans). With the help of the Neo4j-own query language Cypher and its extension it is possible to make queries to the database. For the extraction of the data from the information base (BIM models) and the formulation of the query (semantic fingerprints) a formal description based on GraphML is developed and implemented. A graph corresponds to the representation of a floor of a building. The models of the semantic fingerprints and the AGraphML (Architectural GraphML) specification provide the necessary basis for the structuring and designation of nodes and edges of a graph, where AGraphML represents information about spatial and topological relations as XML.

Different methods of semantic retrieval are developed and implemented for the implementation of the graph-based similarity comparison using semantic fingerprints. The methods can be divided into (sub-)graph-based and case-based. The (sub-)graph-based methods are an adapted implementation of VF2 (an exact graph isomorphism method) and two index-based subgraph isomorphism methods: Index-Based Retrieval with direct queries to a graph-based database and GML Matcher ¹². (Figure 5).

Figure 5.Rule-based coordinator.

Case-based methods include the retrieval approach MetisCBR ¹³, which is based on a distributed, ontologically supported search procedure. All methods support the determination of semantic fingerprints from the query and the application of these fingerprints to the designs in the respective database for the purpose of finding most similar designs. Furthermore, for MetisCBR special similarity measures are defined ¹⁴ for the derived fingerprints, e.g. for topological information of the notification, the deviations of the number of nodes and edges or for proportions the deviations of the area difference of two spaces. (Figure 6).

Figure 6.MetisCBR overview (figure adapted from 13).

The retrieval methods developed are subjected to an automated performance test, which is described in ¹⁵ and includes a rough estimation of the memory requirement as well as a measurement of the runtime. In this test, all approaches showed a basic readiness and remained within acceptable runtime and memory requirements, with visible differences due to the diversity of approaches. MetisCBR and GML Matcher (as one of the available methods) are also tested in ¹⁶ with a performance analysis. This showed the expected behavior where MetisCBR provided more results on almost every request. However, due to the exact nature of GML Matcher's matching algorithm, it had higher values with average similarity values.

A further investigation of the quality of query mechanisms using different user scenarios is carried out. The investigated coordination approaches are the rule-based middleware service KSD Coordinator ¹⁷ (Figure 4) and the rule- and case-based coordination component of the retrieval approach MetisCBR described in ¹³. Subjectively evaluated are, according to a questionnaire, the result sets for two types of queries: contiguous (i.e. assigned to a common scenario) and independent (each query represents its own scenario).

In the result of the first user study slight superiority of the exact approach was found ¹⁵. Subjectively, this provided better results for a majority of 10 queries. The queries were evaluated by the participants according to quality, i.e. according to subjectively perceived similarity and potential to be helpful. The results of the second user study have shown that both approaches are suitable for certain scenarios and goals of retrieval: the KSD coordinator for those scenarios that aim to match the (partial) structures of the query as closely as possible, and the MetisCBR coordinator for those queries that are more oriented towards exploration and search for inspiration ¹³.

All examined approaches use AGraphML as exchange format for transmission of queries. A special protocol is developed on top, which contains the AGraphML of the query, the information about the fingerprints and weightings in the form of XML elements. In the case-based approach Metis CBR ¹³, information available in AGraphML is extracted and transformed into an attribute-value representation for storing and retrieval in the case base. Furthermore, an ontological representation of the AGraphML is used for inner transfer within the system. The KSD coordinator implements a query protocol containing detailed query conditions and A Graph ML representation as XML elements. The GML Matcher expects the A Graph ML request in the data parameters of an HTTP POST request. The index-based retrieval includes an implementation where Cypher queries are created from the query protocol.

The aim of the models is to provide IT support for the early phases of the design process in order to be able to efficiently research similar architectural situations as inspiration, concrete solutions or to identify the actual problem. The iterative nature of the design process requires a continuous shift between creative, analytical and testing steps to select the most appropriate design approach by selecting the most promising variants. For the computer-based support of these visually influenced ways of thinking, semantic fingerprints are developed and derived as topological metadata comparable to the identification of a human being on the basis of his fingerprint. The fingerprints are represented as a graph and used as a subgraph of the entire graph of the building model for search algorithms.

Design ideas are formalized in a multi-stage procedure. The design ideas are described in sketches and topological correlations of the sketches are formalized computer-based in graphs. With the help of these graphs search queries are made, the found hits serve either as inspiration for design ideas or for their concretization or as feedback for a change of the search queries. Several methods have been designed, implemented and evaluated to identify hits that represent similar designs. These include both classical (sub-)graph matching methods and adapted case-based approaches based on current CBR techniques. In addition, direct database queries in a graph database are applied. In the evaluations these retrieval methods showed a basic functionality.

Which characteristics an information system should have for the architectural design in the early phases is determined with a heuristic evaluation (see Figure 7). The 'Importance - Satisfaction Diagram' ³ shows which properties a thinking tool should have in the early design phases and how well the properties can be implemented in the described research scenario. To this end, seven young architects are asked to indicate the importance of certain properties from their point of view, in order to then research them using freehand drawings. Subsequently, a second questionnaire is used to assess the extent to which the properties appear feasible.

Figure 7.Evaluation of the use case

Of the 24 properties surveyed (Figure 7), 17 (green range) are rated as rather good to very good to implementable and further development is recommended. The highest priorities for further development are two-dimensional representations (4.5), drawing design ideas (4.2) and freehand drawings (4.5). Only the implementation of schematic representations (4.1) requires improvements (red area) and must be revised. The goal of future user surveys must be an evaluation of property 4.1 in the green area. Properties in the blue area have a low priority for follow-up by the architects. Building Information Models (BIM) are given the lowest priority to support the early design phases as not important and not easily implementable.