Beat estimation from musician visual cues
Yükleniyor...
Tarih
2021
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
Sound and Music Computing Network
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Özet
Musical performance is an expressive art form where musicians interact with each other using auditory and nonverbal information. This paper aims to discover a robust technique that can identify musical phases (beats) through visual cues derived from a musician’s body movements captured through camera sensors.A multi-instrumental dataset was used to carry out a comparative study of two different approaches: (a) motiongram, and (b) pose-estimation, to detect phase from body sway. Decomposition and filtering algorithms were used to clean and fuse multiple signals. The final representations were analysed from which estimates of the beat, based on a’trust factor’, were obtained. The Motiongram and pose estimation were found to demonstrate usefulness depending on the musical instrument as some instrument playing gestures stimulate more movement in the players than others. Overall, the results were most promising using motiongram. It performed well where string instruments were used. The spatial derivative technique based on human pose estimation was consistent with woodwind instruments, where only a small degree of motion was observed. Copyright: © 2021 the Authors.
Açıklama
Elk
18th Sound and Music Computing Conference, SMC 2021 -- 29 June 2021 through 1 July 2021 -- Virtual, Online -- 175590
18th Sound and Music Computing Conference, SMC 2021 -- 29 June 2021 through 1 July 2021 -- Virtual, Online -- 175590
Anahtar Kelimeler
Body Movements, Body Sway, Camera Sensor, Comparatives Studies, Decomposition Algorithm, Musical Performance, Non-Verbal Information, Pose-Estimation, Robust Technique, Visual Cues, Music
Kaynak
Proceedings of the Sound and Music Computing Conferences
WoS Q Değeri
Scopus Q Değeri
N/A
Cilt
2021-June
