A novel approach

We present a novel teaching tool for structural mechanics and other STEM subjects such as physics, mechatronics and dynamics. This novel teaching aid consists of two major innovations:

• 1) Physical components that closely resemble the graphic symbols used in their pedagogy and can be joined together into assemblies representing (e.g. structural) systems that can, in turn, be physically interacted with by the user(s).
• 2) Automatic or scripted generation of digital twins of the user-made assemblies for which a simulation depicts the same deflections as induced by the user; they can optionally display additional information (e.g. internal forces or enhancing visual cues) via a projected image, thereby augmenting the learning experience.

Educational software programs commonly fail to convey important information in an accessible, fun or effective way. Furthermore, not all educational software programs exploit the benefits of the ‘haptic bond’ between visual and auditory stimuli, which has well-established cognitive and pedagogical value (Fredembach, de Boisferon, & Gentaz, 2009). The physical teaching props that do (e.g. sponge blocks, hanging chains) are generally prohibitively impractical and narrow in their scope of concepts covered. The innovations presented here promise to deliver potentially disruptive technological solutions in the space of educational tools for STEM subjects, a field so fundamental and pervasive that almost every child on the planet encounters it. The general concept of this novel method (A+Ha!) is to facilitate both free and scripted learning, enhanced by haptic interaction and augmented via digital simulations. This patent-protected technology aims to make the learning of STEM concepts more effective, more accessible and more fun. The embodiments described in this chapter are representative of a fully functioning prototype of the A+Ha! system. It should be noted that significant further developments are planned and therefore not fully represented by this early, albeit fully functional, prototype.

Physical components

The main physical components of the novel system comprise

• • a back panel that doubles as a projection surface featuring an array of crossshaped orifices to facilitate the docking of physical nodal components,
• • physical nodal components such as pinned or rigid connectors as well as various types of supports that can be docked at 90° orientations into the back panel by means of an elastic cross-shaped buckle and
• • physical linear components such as beams, struts and cables that connect the nodal components.

In contrast to the pre-set configurations of typical lab equipment (e.g. from GUNT or TecQuipment), this system grants the user to engage in explorative design with the freedom to generate a plethora of assemblies, to experiment with them physically and to iteratively explore variations thereof, thereby broadening understanding of structural behaviour (Figure 11.4). Such an explorative task could, for example, involve the student being prompted to assemble a reticulated frame (i.e. such that it is unstable and collapses as a mechanism) and then the student is tasked with finding ways to stabilise the frame by adding bracing or by changing support conditions and connections, see Figure 11.3. The solutions that the student finds can of course be interacted with physically and augmented digitally.

A key innovation is that the physical components closely resemble their schematic symbols used in pedagogy. In structural mechanics, this relates to the geometric shapes (triangles, circles and lines) used to represent such items as pinned or rigid connections as well as pinned, sliding or rigid supports — see Figure 11.5.

Digital augmentation

The digital augmentation software can be divided into two primary functions:

• 1. A physics solver that simulates deflections and resultant forces of the digital twin
• 2. An interface that facilitates user interaction with the physical construct and the digital augmentation synchronously

By monitoring how the finite elements deform under user-induced deformations, internal forces can be generated and plotted (Figure 11.6).

Figure 11.3 The physical components of the system are assembled to represent (e.g. structural) systems that are removably docked into a back panel-cum-projection surface. Here the user is iteratively exploring various ways to stabilise a frame.

Figure 11.4 A plethora of custom assemblies can be created by the user facilitating explorative design and problem solving.

Figure 11.5 A key innovation is that the physical components closely resemble their schematic symbols used in pedagogy.

Dynamic relaxation physics simulation

A fundamental innovation of this technology is simulating a digital twin of the physical assembly that deforms synchronously with it upon user-induced deformation. This has been achieved with an underlying physics simulation using a projection-based dynamic relaxation solver; initial prototypes were completed using Kangaroo (Piker, 2013).

The dynamic relaxation (DR) method for solving physical simulations was first developed in the 1960s (Day, 1965) and has since become established in

Figure 11.6 Four different assemblies displaying four different layers of augmentation. From top left clockwise: reactions, axial forces, shear forces & bending moments.

many fields. In the built environment, DR is most commonly associated with stiffness-independent (purely force based) membrane form-finding. In structural engineering, implicit integration methods are more common that can accurately describe mechanical stresses and displacements in a discretised continuum under the assumption of small deflections. If equilibrium is being sought in a system where deflections are large, the stiffness matrix must be updated over multiple iterations in a non-linear analysis that can be computationally demanding and often unstable. Generally, a prerequisite for implicitly integrated methods is that the systems must be statically determinate or indeterminate. Mechanisms can cause numerical instability and are more difficult to solve. DR on the other hand does not require the computation and inversion of a global stiffness matrix, but instead seeks equilibrium in each node explicitly and simultaneously by assigning mass, acceleration and a method of damping to the nodes. As highlighted by Martini (2006), this means that DR methods are insensitive to the static determinacy of the structural system such that mechanisms and large deformations are not an issue, provided the solver is able to remain stable. This insensitivity to static determinacy and large deformations is highly suitable and forgiving in an experimental and interactive design environment such as this one. Imposing large deformations on the fly and observing the immediate impact they have on structural performance (e.g. deflections, internal forces and reactions) is an illuminating experience for the user.

In order to achieve a digital twin that behaves synchronously with the physical construct, the following technical features were developed:

• • The physical components are represented by corresponding objects and matching topological connectivity in the digital twin (see Figure 11.7 left).
• • The discretisation of the finite elements is similarly embodied in the physical components (see Figure 11.7 right).

Matching the design of the physical components with the discretisation logic of the underlying finite element model presents two significant advantages:

• • The matching ensures a close correlation between the digital and physical deflections. In effect, this means that the plotting of internal forces per finite element corresponds to the segmentation of the elements (i.e. beam, strut or cable).
• • For more advanced users, the matching provides an added pedagogical value by providing a physical abstraction of finite element discretisation, a fundamental aspect of structural engineering analysis.

Previous and less advanced prototypes of this technology (Quinn, Galeazzi, & Gengnagel, 2017; Quinn, Geleazzi, Schneider, & Gengnagel, 2018) were implemented with a DR solver based on three degrees of freedom and subsequently were handicapped in the types of systems and constraints that could be modelled. The prototype presented here exploits finite elements

Figure 11.7 Left: The object bodies as defined by the digital twin correlate with the physical components. Right: the finite element discretisation of the digital twin is also replicated in the physical components.

featuring six degrees of freedom, without which the accurate representation of rigid supports (and more) would not be feasible.

User interface

Since users interact with both the physical construct and the digital twin, this presents some interesting interface design challenges. There should be a consistent and fluid duality between the physical assembly and its digital twin, not just visually but also through sensory interactions.

The working prototype allows the user to navigate a basic digital UI to select one of the exemplary learning objectives. Once a learning objective has been selected, the user is tasked with building a structural assembly, which is presented graphically by means of the relevant schematic symbols (Figure 11.8). Further augmentative information can be added to the UI, such as images referencing real-world equivalent structures or embedded data on material performance or cost. This ability to customise and carefully curate the amount and type of complementary data or visuals is a unique strength of augmented reality.

Wand & tracking

User interaction with the physical components must be emulated precisely in the simulation of the digital twin. To achieve this, a physical hand-held wand device was developed (Figure 11.9), which ensures that the user interacts with the physical components in a way that can be replicated in the simulation. This was realised with two separate modes:

Figure 11.8 The system facilitates pedagogically scripted learning objectives (e.g. with 'build me’ scenarios).

Figure 11.9 The hand-held wand ensures simple and deliberate interaction with the physical components; this is accurately replicable in the simulation. The wand is tracked via IR LED.

1) Beam push mode

When physically interacting with beam elements, the wand is equipped with a spinning wheel attachment. This wheel ensures an effectively frictionless collision with the beam element, which is duplicated in the simulation by means of a collision force between a circular object and the segments of the beam.

2) Node pull mode

When interacting with strut assemblies such as trusses, it is not desirable or effective for the user to push the struts along their lengths; it is better to pull at the nodes and to observe the induced compression and tensions in the members. For this mode, the wheel is simply removed from the wand revealing a peg that is the right size to dock inside a cylindrical cavity in every pinned nodal component. The simulation replicates this behaviour by applying an attraction force to the node in question.

The location of the wand is tracked by means of infrared LEDs embedded in the wand and computer vision algorithms using a camera that filters out visible light. The tracking is computationally lightweight, robust and fast. To facilitate interface navigation, the wand can be used as a point-and-click device. This essentially turns the board into a big touchscreen. An important and novel feature of the interface design is its ability to augment sensory experience; it does so by displaying the actual magnitude of force the user exerts on the structure promoting intuition for what a unit of force actually ‘feels’ like. The authors are also investigating object recognition on the fly, removing the need for a hand-held wand altogether.