Chakraborty, SutirthaAktaş, SenemClifford, WilliamTimoney, Joseph2024-09-252024-09-252021978-889454154-02518-3672https://hdl.handle.net/20.500.12491/12307Elk18th Sound and Music Computing Conference, SMC 2021 -- 29 June 2021 through 1 July 2021 -- Virtual, Online -- 175590Musical performance is an expressive art form where musicians interact with each other using auditory and nonverbal information. This paper aims to discover a robust technique that can identify musical phases (beats) through visual cues derived from a musician’s body movements captured through camera sensors.A multi-instrumental dataset was used to carry out a comparative study of two different approaches: (a) motiongram, and (b) pose-estimation, to detect phase from body sway. Decomposition and filtering algorithms were used to clean and fuse multiple signals. The final representations were analysed from which estimates of the beat, based on a’trust factor’, were obtained. The Motiongram and pose estimation were found to demonstrate usefulness depending on the musical instrument as some instrument playing gestures stimulate more movement in the players than others. Overall, the results were most promising using motiongram. It performed well where string instruments were used. The spatial derivative technique based on human pose estimation was consistent with woodwind instruments, where only a small degree of motion was observed. Copyright: © 2021 the Authors.eninfo:eu-repo/semantics/closedAccessBody MovementsBody SwayCamera SensorComparatives StudiesDecomposition AlgorithmMusical PerformanceNon-Verbal InformationPose-EstimationRobust TechniqueVisual CuesMusicConference Object2021-June46522-s2.0-85122096528N/A