Published Papers

My research develops through exhibitions and artistic production. Exhibitions function not as presentation venues but as sites of inquiry, where theoretical frameworks are implemented as concrete systems, tested in real contexts, and refined through successive iterations.

A Hybrid Approach

Conventional disciplinary divisions assign responsibility by domain: art addresses aesthetics, engineering handles implementation, science verifies results. Each discipline offers distinct strengths and methodological rigour. Yet such separation proves limiting when working with complex systems, which exhibit properties that no single discipline can adequately address.

Complex systems are characterised by emergence, feedback, and nonlinearity. These are behaviours that arise from interaction rather than from isolated components. Theory transforms during implementation, unexpected phenomena emerge during operation, and user interaction reshapes meaning in ways that cannot be anticipated through design alone. From this perspective, maintaining strict disciplinary boundaries forces a choice between partial accounts. Either prioritise aesthetic coherence and lose technical precision, or optimise system performance and sacrifice experiential depth.

My work refuses this choice. Rather than maintaining disciplinary separation, it operates across art, technology, and science simultaneously, treating each as necessary but insufficient on its own.

Consider human-system interaction. Examined scientifically, it becomes data input, processing, and output, which provides measurable precision but risks reducing experience to mechanism. Philosophically, it concerns relationships between observer and observed, which opens conceptual depth but may lack concrete application. Artistically, it addresses the quality of experience and modes of participation, which centres embodied encounter but may resist systematic analysis. Ethically, it asks who is affected and what remains excluded, which foregrounds justice but can complicate creative experimentation. None of these perspectives is complete. Each reveals blind spots in the others.

Designing systems from this integrated position produces emergent forms of interaction impossible within any single framework. A system optimised purely for accuracy differs fundamentally from one optimised for experiential resonance. Working across these dimensions generates new behaviours where accuracy and resonance become mutually constitutive rather than competing values. This is both the method and the aim of the research.

Core Research Questions

My research explores the following questions:

Can cognition be understood as distributed across human-machine-environment networks?

How might intelligent systems be designed when cognition is understood as fundamentally relational rather than individual? Hutchins (1995) provides the theoretical foundation through distributed cognition, which demonstrates that cognitive processes extend across external artifacts, social interactions, and environmental structures rather than remaining bounded within individual minds. Building on this framework: if cognition operates through relational networks, what forms of human-system interaction become possible?

Can data be treated as dynamic behaviour rather than static representation?

Rather than treating data as fixed values awaiting analysis, what happens when data is understood as processes with their own behaviours? Gitelman (2013) establishes that data is never raw or neutral but always shaped by the conditions of its collection and the frameworks through which it becomes legible. Extending this insight: if data is relational and performative, capable of negotiation and adaptation, how does this transform system design?

How do relational networks produce emergent system behaviour?

How might systems be designed as relational networks rather than assemblies of discrete parts? Bonabeau (2002) demonstrates through agent-based modelling that emergent phenomena arise from local interactions rather than global coordination, exhibiting feedback loops, nonlinear dynamics, and self-organization. If systems are designed around relational dynamics rather than component assembly, what new forms of interaction and emergence become possible?

Can indeterminate and transitional states be modelled without forcing resolution?

Can computational systems model indeterminacy, ambiguity, and process without collapsing them into fixed categories? While affective computing has conventionally treated emotion as discrete states, recent work by Metallinou et al. (2012) and Liu et al. (2016) shows that emotion operates within continuous spaces where states blend and transition. Riley (2012) extends this to cognition more broadly, describing it as a temporally extended dynamic process with pre-articulate phases and post-reflective residues. How might we model the moments where meaning begins to form without forcing premature resolution?

Frameworks and Research Direction

These questions have shaped the development of three interconnected frameworks. Inquiry into distributed cognition and relational emergence produced the Relational Cognitive System (RCS). Exploration of data as process rather than object led to Behaviour-Driven Systemic Sonification (BDSS). Study of emotion as temporally unfolding rather than discretely classified resulted in the Pre-Verbal Affective Dynamics (PVAD) system.

Research Method: Theory Through Implementation

Theory in this practice does not remain abstract but translates into working systems. Each implementation tests frameworks within specific contexts, generating insights that feed back into theoretical development.

The Plant Photosynthesis Sonification experiment marked the first BDSS implementation. Environmental variables were mapped into vector spaces, plant states into attractor zones, adaptive processes into agent negotiations. This model introduced global system awareness, treating data as active agents rather than passive values.

Embodied Algorithms extended BDSS into interactive installation. Industrial hot-rolling data was reinterpreted as a relational system, integrating the embodied expertise of a POSCO master engineer into a multimodal interface. Real-world application made the method more concrete.

Resonance Loop approached emotion not as discrete states but as continuous temporal processes. PVAD transformed 468 facial landmarks into a three-dimensional meta-coordinate system representing temporal dynamics, classified into six narrative phases. "Pre-Verbal Hesitation" and "Post-Verbal Residue" formed the system's core, exploring how unresolved emotional states could be modelled without closure. The recursive feedback loop, where generated voice becomes new input, realised RCS's principle of temporal narration.

Each implementation generates new theoretical questions. Unexpected system behaviour signals opportunity to revise frameworks. Unforeseen audience responses produce new forms of interaction design. Emergent behaviour indicates that relational dynamics function as intended.

Exhibitions are not endpoints but developmental phases. Each implementation raises new questions. Each question leads to new implementation. Theory and practice, research and creation, question and response remain inseparable.

References

Bonabeau, E. (2002). Agent-based modeling: Methods and techniques for simulating human systems. PNAS, 99(suppl 3), 7280-7287.

Gitelman, L. (Ed.). (2013). "Raw data" is an oxymoron. MIT Press.

Hutchins, E. (1995). Cognition in the wild. Cambridge, MA: MIT Press.

Liu, W., Zheng, W. L., & Lu, B. L. (2016). Emotion recognition using multimodal deep learning. In International Conference on Neural Information Processing (pp. 521-529). Springer.

Metallinou, A., Wollmer, M., Katsamanis, A., Eyben, F., Schuller, B., & Narayanan, S. (2012). Context-sensitive learning for enhanced audiovisual emotion classification. IEEE Transactions on Affective Computing, 3(2), 184-198.

Riley, M. A. (2012). Dynamics of cognition. Wiley Interdisciplinary Reviews: Cognitive Science, 3(6), 593-606.