基于手部動作感應和即時音頻合成的虛擬演奏系統(tǒng)

        我們以EC5-1719CLDNA平臺為基礎開發(fā)了一套虛擬的音樂演奏系統(tǒng)。所謂虛擬，是指這套系統(tǒng)展示在你面前的只有一個虛擬的場景，而在你手中的也不再是沉重的吉他、笛子，而只有一副手套。這副手套允許你在空氣中進行樂器的演奏。

        系統(tǒng)從音樂演奏的客觀需求出發(fā)，綜合考慮各種樂器的演奏特點，提出了吉他、鋼琴、笛子三種類型樂器在同一雙手套上實現(xiàn)的“一體化”演奏方案。該方案利用多種傳感器和機器視覺對手部動作和位置進行感應，并輔以ZigBee模塊實現(xiàn)了信號的無線傳輸。系統(tǒng)還發(fā)揮了平臺雙核處理器的并行處理優(yōu)勢和多媒體功能，在Visual Studio 2005下利用DirectX SDK開發(fā)了它的核心部分——3D演奏場景以及音頻合成引擎。系統(tǒng)最終實現(xiàn)了自由演奏，樂譜引導下演奏以及演奏過程中樂譜實時創(chuàng)作的功能。此外系統(tǒng)還加入了基于TCP/IP協(xié)議的網(wǎng)絡功能，完成了樂譜的遠程創(chuàng)作及傳輸。

We have developed a set of hypothesized music performance system which takes the EC5-1719CLDNA platform as the foundation. It is called hypothesized, because this set of system display a hypothesized scene for the users. There will not be any heavy guitar or bamboo flute but just a pair of gloves for playing.

Embarking from the demand of music performance, considering musical instruments’ performance characteristic, the system proposed an integration performance plan which realizes guitar, piano and bamboo flute with just a pair of gloves. This plan utilizes kinds of sensors and computer vision to induce hand movement, as well as hand position. What is more, wireless transmission of signals is carried out by the employment of ZigBee module. The plan has also fully displayed Dual-core processors' parallel processing superiority and the multimedia functions of the platform. It develops the 3D performance scene and audio synthesis engine, using Visual Studio 2005 and DirectX SDK. The system finally realizes free performance function, performance under music guidance and real-time score creation function. In addition, with network interconnection based on TCP/IP protocol, the system possesses long-distance music score creation and transmission function.


linux操作系統(tǒng)文章專題:linux操作系統(tǒng)詳解（linux不再難懂）