‘Correlates’ is an interactive immersive digital video projection that explores how people can be visually represented in a manner that evokes how we are rendered as our data doubles within the data extraction industry. With the advent of social media and pervasive user-tracking and data capture, the information gathered about users has become the foundation of an emergent surveillance economy (Shoshana Zuboff has described this as surveillance capitalism). In this context users of social media become the raw product of the surveillance economy, a process managed by the various corporations associated with the data extraction industry. Like all extractive industries, the power dynamic and flows of value are one way, and unsustainable.

Users of social media are no longer social subjects but informational objects. In the data extraction industry the data representations of users are known as correlates – abstractions of individual users correlated with other users, mapping relationships in order to ascertain and predict user behaviour, through a process known as sentiment analysis. These predictions are the vehicle of exchange in the industry – what is bought and sold. It is valuable to be able to predict the future. Beyond that, British neuroscientist Anil Seth has proposed that the capacity to predict is the foundation of human consciousness, raising the possibility that predictive AI based systems may yet acquire similar qualities.

The Correlates project employs full body tracking (using the Microsoft Azure Kinect IR sensor), which calculates the physical location and attitude of ‘interactor’ (viewer’s) bodies, creating (invisible within the work) skeletal representations of interactors in the projected 3D space. Live video captured of the interactors occupying the installation space is mapped in real-time to these ‘skeletons’ so as to create three dimensional diffracted informational images (grates) of the interactors, which are then visible in the video projections around them. These images are formed from the arrays of pixels in the live video image, rendered as various kinds of ‘diffraction gratings’ (patterns or arrays of pixels), forming overlaying interference patterns that are activated by the parallax effects of the movement of the interactors. At times the virtual camera, which determines the point of view and what is within frame, shifts from a third person objective position to a first person subjective position (attached to a specific interactor), generating an uncanny situation where interactors are able to see themselves from another’s point of view.

Interactors within the interactive installation are immersed in complex visual structures that appear as multi-layered and wave- like iridescent three dimensional patterns. The effect is not dissimilar to the holographic space of an interferogram, that manifests as a dense informational space where interactors can discern visual traces of their appearance – faces, hands, clothes, bodies, etc – and interactions with one another and the imagery that surrounds them.

‘Correlates’ explores the constructed nature of reality as generated through interaction between multiple interactors in an interactive installation space and a representational logic that observes these interactors not as subjects but as data objects and relational vectors (social relations reduced to correlates). The resulting spatialisation and visualisation appears as shifting iridescent informational patterns, resembling a reality constructed as moire patterns of shifting relations, in which visual and movement traces of interactors within the installation are apparent.