Page 186 - My FlipBook
P. 186
研
人 Assistant Research Fellow
員 蘇黎 Li Su
Faculty Ph.D., Graduate Institute of Communication Engineering,
National Taiwan University, Taiwan
T +886-2-2788-3799 ext.1806 E lisu@iis.sinica.edu.tw
F +886-2-2782-4814 W www.iis.sinica.edu.tw/pages/lisu/
・ Best Paper Award of the International Society of Music Information Retrieval Conference
(ISMIR), Delft, the Netherlands (2019)
・ Assistant Research Fellow, Institute of Information Science, Academia Sinica, Taiwan
(2017-present)
・ Postdoctoral Research Fellow, Center for Information Technology Innovation, Academia
Sinica, Taiwan (2012-2016)
・ Ph.D., Graduate Institute of Communication Engineering, National Taiwan University, Taiwan
(2008-2012)
・ B.S. (Double Degree), Electrical Engineering & Mathematics, National Taiwan University,
Taiwan (2003-2008)
Research Description
My research interests focus on machine musicianship, the dream of making a machine understand music as a musician does. Research
areas for this interdisciplinary topic include automatic music transcription, machine music appreciation, automatic music generation,
and computational creativity of music. Applications are found in music production, education, entertainment, well-being, computational
musicology, revitalization of music cultural heritage, and others related to next-generation music industry. Research directions are two-fold:
the rst is the analysis of scores written by a composer (i.e., automatic music transcription), while the second is the analysis of interpretation
in musical performance (i.e., music expression analysis).
The core technology I have developed over the past three years is a system for automatic music transcription. Automatic transcription of
polyphonic music is the holy grail in machine listening. I proposed a novel method, called the combined frequency and periodicity (CFP)
method, which detects pitches according to the agreement of a harmonic series in the frequency domain and a subharmonic series in the
lag domain. This approach nicely aggregates the complementary advantages of the two feature domains in di erent frequency ranges, and
improves the robustness of the pitch detection function to the interference of the overtones of simultaneous pitches.
As an ongoing project, my recent e ort of research is focused on developing virtual musicians, and making them understand the music
content, behave with music in reasonable body movements, and even play music with real human. The main research topic to achieve this
goal is to develop solutions to jointly generate multi-modal contents including image, motion, audio and a ective information directly from
music repertoire. Up to date, we have collected a music dataset containing high-quality music semantic annotation and body movement,
and proposed a music-to-body-movement generation framework. The developed virtual musician technologies will add great possibilities in
the industry of animation and computer-human interaction.
Publications 5. Yu-Te Wu, Berlin Chen, Li Su, "Polyphonic Music Transcription
with Semantic Segmentation," IEEE International Conference on
1. Jun-Wei Liu, Hung-Yi Lin, Yu-Fen Huang, Hsuan-Kai Kao, Li Su, Acoustics, Speech and Signal Processing (ICASSP), May 2019.
"Body movement generation for expressive violin performance
applying neural networks," IEEE International Conference on 6. Chen-Yun Lin, Li Su, and Hau-tieng Wu, "Wave-shape function
Acoustics, Speech and Signal Processing (ICASSP), May 2020. analysis - when cepstrum meets time-frequency analysis," Journal
of Fourier Analysis and Applications (JFAA), volume 24, number
2. Yu-Fen Huang, Tsung-Ping Chen, Nikki Moran, Simon Coleman, 2, pages 451-505, April 2018.
and Li Su, "Identifying expressive semantics in orchestral
conducting kinematics," International Society of Music 7. Li Su and Yi-Hsuan Yang, "Combining Spectral and Temporal
Information Retrieval Conference (ISMIR), November 2019. Representations for Multipitch Estimation of Polyphonic Music,"
IEEE/ACM Trans. Audio, Signal Language Proc. (TASLP) ,
3. Zih-Sing Fu and Li Su, "Hierarchical classification networks volume 23, number 10, pages 1600-1612, October 2015.
for singing voice segmentation and transcription," International
Society of Music Information Retrieval Conference (ISMIR), 8. Li Su, Chin-Chia M. Yeh, Jen-Yu Liu, Ju-Chiang Wang, and
November 2019. Yi-Hsuan Yang, "A systematic evaluation of the bag-of-frames
representation for music information retrieval," IEEE Trans.
4. Tsung-Ping Chen and Li Su, "Harmony transformer: incorporating Multimedia (TMM) , volume 16, number 5, pages 1188-1200,
chord segmentation into harmony recognition," International August 2014.
Society of Music Information Retrieval Conference (ISMIR),
November 2019, (Best Paper Award)
184
人 Assistant Research Fellow
員 蘇黎 Li Su
Faculty Ph.D., Graduate Institute of Communication Engineering,
National Taiwan University, Taiwan
T +886-2-2788-3799 ext.1806 E lisu@iis.sinica.edu.tw
F +886-2-2782-4814 W www.iis.sinica.edu.tw/pages/lisu/
・ Best Paper Award of the International Society of Music Information Retrieval Conference
(ISMIR), Delft, the Netherlands (2019)
・ Assistant Research Fellow, Institute of Information Science, Academia Sinica, Taiwan
(2017-present)
・ Postdoctoral Research Fellow, Center for Information Technology Innovation, Academia
Sinica, Taiwan (2012-2016)
・ Ph.D., Graduate Institute of Communication Engineering, National Taiwan University, Taiwan
(2008-2012)
・ B.S. (Double Degree), Electrical Engineering & Mathematics, National Taiwan University,
Taiwan (2003-2008)
Research Description
My research interests focus on machine musicianship, the dream of making a machine understand music as a musician does. Research
areas for this interdisciplinary topic include automatic music transcription, machine music appreciation, automatic music generation,
and computational creativity of music. Applications are found in music production, education, entertainment, well-being, computational
musicology, revitalization of music cultural heritage, and others related to next-generation music industry. Research directions are two-fold:
the rst is the analysis of scores written by a composer (i.e., automatic music transcription), while the second is the analysis of interpretation
in musical performance (i.e., music expression analysis).
The core technology I have developed over the past three years is a system for automatic music transcription. Automatic transcription of
polyphonic music is the holy grail in machine listening. I proposed a novel method, called the combined frequency and periodicity (CFP)
method, which detects pitches according to the agreement of a harmonic series in the frequency domain and a subharmonic series in the
lag domain. This approach nicely aggregates the complementary advantages of the two feature domains in di erent frequency ranges, and
improves the robustness of the pitch detection function to the interference of the overtones of simultaneous pitches.
As an ongoing project, my recent e ort of research is focused on developing virtual musicians, and making them understand the music
content, behave with music in reasonable body movements, and even play music with real human. The main research topic to achieve this
goal is to develop solutions to jointly generate multi-modal contents including image, motion, audio and a ective information directly from
music repertoire. Up to date, we have collected a music dataset containing high-quality music semantic annotation and body movement,
and proposed a music-to-body-movement generation framework. The developed virtual musician technologies will add great possibilities in
the industry of animation and computer-human interaction.
Publications 5. Yu-Te Wu, Berlin Chen, Li Su, "Polyphonic Music Transcription
with Semantic Segmentation," IEEE International Conference on
1. Jun-Wei Liu, Hung-Yi Lin, Yu-Fen Huang, Hsuan-Kai Kao, Li Su, Acoustics, Speech and Signal Processing (ICASSP), May 2019.
"Body movement generation for expressive violin performance
applying neural networks," IEEE International Conference on 6. Chen-Yun Lin, Li Su, and Hau-tieng Wu, "Wave-shape function
Acoustics, Speech and Signal Processing (ICASSP), May 2020. analysis - when cepstrum meets time-frequency analysis," Journal
of Fourier Analysis and Applications (JFAA), volume 24, number
2. Yu-Fen Huang, Tsung-Ping Chen, Nikki Moran, Simon Coleman, 2, pages 451-505, April 2018.
and Li Su, "Identifying expressive semantics in orchestral
conducting kinematics," International Society of Music 7. Li Su and Yi-Hsuan Yang, "Combining Spectral and Temporal
Information Retrieval Conference (ISMIR), November 2019. Representations for Multipitch Estimation of Polyphonic Music,"
IEEE/ACM Trans. Audio, Signal Language Proc. (TASLP) ,
3. Zih-Sing Fu and Li Su, "Hierarchical classification networks volume 23, number 10, pages 1600-1612, October 2015.
for singing voice segmentation and transcription," International
Society of Music Information Retrieval Conference (ISMIR), 8. Li Su, Chin-Chia M. Yeh, Jen-Yu Liu, Ju-Chiang Wang, and
November 2019. Yi-Hsuan Yang, "A systematic evaluation of the bag-of-frames
representation for music information retrieval," IEEE Trans.
4. Tsung-Ping Chen and Li Su, "Harmony transformer: incorporating Multimedia (TMM) , volume 16, number 5, pages 1188-1200,
chord segmentation into harmony recognition," International August 2014.
Society of Music Information Retrieval Conference (ISMIR),
November 2019, (Best Paper Award)
184
