Görüntüden müziğe dönüşüm: Yazılım aracılığıyla dijital görüntüden müzik materyali elde etme
The conversion from image to music: Obtaining musical material from digital images through software
- Tez No: 865743
- Danışmanlar: DOÇ. DR. ERAY ALTINBÜKEN, DR. ÖĞR. ÜYESİ OZAN SARIER
- Tez Türü: Yüksek Lisans
- Konular: Müzik, Music
- Anahtar Kelimeler: Belirtilmemiş.
- Yıl: 2024
- Dil: Türkçe
- Üniversite: İstanbul Teknik Üniversitesi
- Enstitü: Lisansüstü Eğitim Enstitüsü
- Ana Bilim Dalı: Müzikoloji ve Müzik Teorisi Ana Bilim Dalı
- Bilim Dalı: Müzik Teorisi ve Kompozisyon Bilim Dalı
- Sayfa Sayısı: 111
Özet
Bu tez çalışması, dijital görüntülerin dönüştürülmesini ve bu dönüşümün ardından müzik besteleme sürecinde kullanılabilecek materyaller elde etmeyi hedefleyen bir çalışmadır. Bu hedefe ulaşmak amacıyla yazılım temelli bir yöntem geliştirilmiştir. Müzik besteleme sürecinde motif, melodi, akor gibi müzikal materyaller ortaya çıkar. Bu önemli ögeler yani materyaller müzik kompozisyonu dahilinde belirli bir anlam bütünlüğü sağlamak ve bir ses tasarımı oluşturmak için kullanılır. Bu çalışma resim ve görüntüleri müzikal materyale dönüştürerek besteleme sürecinde kullanılabilecek materyaller elde etmek için yapılmıştır. Python programlama dili kullanılarak geliştirilen bir dizi algoritma ile dijital resim veya görüntüler MIDI aracılığıyla sese dönüştürülmüştür. Bu yöntemle elde edilen müzikal materyaller, besteleme sürecinde yeni, özgün ve yaratıcı materyallerin hızlı ve sistemli bir şekilde üretilmesine olanak tanımaktadır. Çalışmada, iki temel hipotez vardır. İlk hipoteze göre, dönüşüm algoritmasıyla işlenen dijital görüntülerden elde edilen çıktılar, tutarlı ve sistematik bir biçimde müzikal materyal üretebilme potansiyeline sahip olmalıdır. İkinci hipoteze göre ise dijital görüntülerden elde edilen çıktılar, besteci müdahalesi sonrasında müzik olarak sunulabilecek niteliğe sahip olmalıdır. Çalışmada bu iki hipoteze yönelik geliştirilen yöntemin müzik besteleme sürecinde etkin bir rol oynayabilecek bir araç haline gelebileceğini göstermek amaçlanmıştır. Yöntem, dijital görüntülerden elde edilebilecek renk, renk parlaklığı, renk doymuşluğu, piksel koordinatı gibi verileri müzik ögeleriyle eşleme üzerine geliştirilmiştir. Müziğin temel bileşenlerinden biri olan nota, görülebilir ışık tayfındaki renklerin frekanslarıyla eşlenip, daha sonra bu renklerin ekranlarda kullanılan RGB renk modeli yaklaşıkları ile kullanılmıştır. Bu işlem ile bir oktav genişliğinde nota eşleri elde edilmiştir. Birden fazla oktav kullanımı için görüntüden elde edilen pikseldeki renk, 256 ton içerecek şekilde kurgulanan monokrom tabloya göre siyah beyaza çevrilmiştir. Bu monokrom dizi siyahtan beyaza piyanonun ses aralığına göre oktavlara bölünmüştür ve böylelikle pikselin siyah beyaz halinin durumuna göre notaya oktav atanmıştır. Dönüşümden elde edilen notaların ses gürlüğü için pikseldeki rengin renk doymuşluğu değeri kullanılmıştır ve MIDI velocity değerine dönüştürülmüştür. Nota süresi hesaplamaları için bir yöntem teoride kurgulanmıştır fakat uygulama aşamasında alınan sonuçlar başarılı sayılacak kadar yeterli olmamıştır. Nota süresi hesaplaması aynı satırda aynı renge sahip ardışık piksellerin bütün olarak tanımlanmasıyla yapılmıştır. Çalışmanın en önemli ve en ayrıştırıcı özelliklerinden biri çıktı sonuçlarının çoksesli olmasıdır. Görüntüler soldan sağa taranırken her bir satır ayrı bir çizgide ilerleyen nota silsilesi olarak düşünülmüştür. Bu da elde edilen çıktılarda teorik olarak görüntünün yüksekliği kadar farklı yatay yapıların oluşabilmesine olanak tanımaktadır. Dönüşümden geçen görüntülerden elde edilen çıktılar incelenirken, bir müzik eserinde bulunabilecek ögelere benzer yapılar aranmıştır. Bunlar; bas partisi, melodik hareketler, akorlar, arpejler gibi ögelerdir. Çıktılarda görülen küme haldeki yapılar incelenerek potansiyel materyalleri ortaya çıkartmak için öneriler de verilmiştir. Elde edilen çıktılarda görüntüden görüntüye miktarları değişkenlik gösterse de bahsi geçen ögeler bulunmuştur ve gösterimi yapılmıştır. Çıktılardaki materyallerin doku içinde işitsel olarak rahat ayırt edilmesi ve müzik olarak kullanılabilirlik potansiyelini azami oranda görebilmek için piyanonun homojen ses yapısı dışında bir şekilde sunulması gerektiği düşünülmüştür ve bu yüzden bir orkestrasyon işlemi de yapılmıştır. Orkestrasyon işlemi yapılırken çıktının orijinal ham haline olabildiğince sadık kalmak için ekstra nota eklenmemiştir. Çalışmanın ikinci hipotezinde müzik olarak sunulmaya uygun durumda olma koşulu vardır. Uygun olma durumu dönüşüm çıktısının el ile işlenebilirliği üzerinedir ve bu konuda nota sayısı önemli bir etken olmuştur. Çalışma içerisinde işlenebilme durumu detaylı şekilde açıklanmıştır. Sonuç olarak ikinci hipoteze belirli kısıtlamalar ve varsayımlarla cevaplar verilmiştir. Elde edilen sonuçların, bilgisayarlı müzik oluşturma ve müzikte yaratıcılık hususunda yeni araştırmalara ve yeni araçlara ilham verme potansiyeline sahip olduğu düşünülmektedir. Bilgisayar bilimi, görsel sanatlar, psikoloji, müzik, eğitim bilimleri gibi alanlar dahil edilerek bu çalışmayı temel alan veya bu çalışmadan ilham alan birçok yeni araştırma ortaya çıkabilir. Araştırmanın gidebileceği potansiyel yönlerden sonuçlar bölümünde bahsedilmiştir.
Özet (Çeviri)
This thesis explores the conversion of digital images to derive materials usable in the process of music composition through developing a software-based method. In the process of music composition, elements such as motifs, melodies, and chords emerge. These crucial components are utilised within the context of the music composition to establish a specific coherence of meaning and to craft a sound design. This study is conducted to convert images into musical materials, with the intention of obtaining materials applicable in the composition process. Using a series of algorithms developed with the Python programming language, digital images are converted into sound through MIDI. This method makes possible the rapid and systematic generation of new, original, and creative materials suitable to be used in the composition process. The study addresses two fundamental hypotheses. According to the first hypothesis, musical materials can be systematically produced from the outputs of digital images processed by the conversion algorithm. According to the second hypothesis, these outputs, possess the qualities required to be presented as music after composer intervention The method developed in this study involves mapping data from digital images, such as colour, brightness, saturation, and pixel coordinates, to musical elements. The fundamental basis of the conversion method lies in the mapping of the frequencies of colours in the visible light spectrum to sound frequencies. Since the study involves digital images, it has been necessary to map the frequencies of light to colour models used in digital environments. For this mapping process, the“Approximate RGB Values for Visible Wavelengths”algorithm created by Dan Bruton in Fortran programming language is employed. Although Bruton's algorithm provides approximate values for the RGB colour model, as the exact equivalence cannot be achieved, it serves the purpose well. After obtaining RGB values, the next step involves mapping these values to sound frequencies. The wavelengths of visible light are converted to frequency values, represented in Terahertz (THz). To ensure that the frequencies are within the audible range, the frequency values are reduced by dividing them by 2 until they are equal to or less than 440 Hz, which corresponds to the fourth octave A note. The lowest frequency value obtained is then matched to the nearest note in the twelve-tone equal temperament system, resulting in a one-octave scale. One noteworthy modification to this algorithm that is done in this study is the blending of colours between adjacent octaves, specifically between the 4th octave F and the 5th octave F. This blending is applied to mitigate the potential harshness in transitions between octaves due to colour differences. The resulting colour is a combination of the individual RGB values, normalised by the total number of colours involved in the blending process.This thesis explores the conversion of digital images to derive materials usable in the process of music composition through developing a software-based method. In the process of music composition, elements such as motifs, melodies, and chords emerge. These crucial components are utilised within the context of the music composition to establish a specific coherence of meaning and to craft a sound design. This study is conducted to convert images into musical materials, with the intention of obtaining materials applicable in the composition process. Using a series of algorithms developed with the Python programming language, digital images are converted into sound through MIDI. This method makes possible the rapid and systematic generation of new, original, and creative materials suitable to be used in the composition process. The study addresses two fundamental hypotheses. According to the first hypothesis, musical materials can be systematically produced from the outputs of digital images processed by the conversion algorithm. According to the second hypothesis, these outputs, possess the qualities required to be presented as music after composer intervention The method developed in this study involves mapping data from digital images, such as colour, brightness, saturation, and pixel coordinates, to musical elements. The fundamental basis of the conversion method lies in the mapping of the frequencies of colours in the visible light spectrum to sound frequencies. Since the study involves digital images, it has been necessary to map the frequencies of light to colour models used in digital environments. For this mapping process, the“Approximate RGB Values for Visible Wavelengths”algorithm created by Dan Bruton in Fortran programming language is employed. Although Bruton's algorithm provides approximate values for the RGB colour model, as the exact equivalence cannot be achieved, it serves the purpose well. After obtaining RGB values, the next step involves mapping these values to sound frequencies. The wavelengths of visible light are converted to frequency values, represented in Terahertz (THz). To ensure that the frequencies are within the audible range, the frequency values are reduced by dividing them by 2 until they are equal to or less than 440 Hz, which corresponds to the fourth octave A note. The lowest frequency value obtained is then matched to the nearest note in the twelve-tone equal temperament system, resulting in a one-octave scale. One noteworthy modification to this algorithm that is done in this study is the blending of colours between adjacent octaves, specifically between the 4th octave F and the 5th octave F. This blending is applied to mitigate the potential harshness in transitions between octaves due to colour differences. The resulting colour is a combination of the individual RGB values, normalised by the total number of colours involved in the blending process. An essential feature of this study is the polyphonic nature of the converted outputs. The resulting polyphonic outputs can be adapted to create diverse compositions suitable for different musical genres and instruments. To facilitate the use and arrangement of polyphonic outputs, the twelve-tone equal temperament system was chosen as the tuning system. This system was selected due to its standardised and widespread use in polyphonic harmonic systems and many Western instruments. However, it's crucial to note that other tuning systems could also be employed. Microtonal systems, for example, could provide access to more intervals and more colours by using smaller steps. Nevertheless, the focus on polyphony led to the initial choice of the twelve-tone equal temperament system. In the previous mapping process, the colours are mapped from the note A to G♯/A♭ within one octave. To allow the use of multiple octaves, it was considered essential for a data obtained from a digital image to play a role in distributing the notes across different octaves. For octave distribution, a relationship is established between the brightness of colours and the octave numbers of notes. For this purpose, a monochromatic scale encompassing black, white, and grayscale tones is created within the RGB colour model. This colour scale, extending from (0, 0, 0) black to (255, 255, 255) white, includes a total of 256 elements or colours and is divided into parts and mapped to octaves. In the first stages of the study, the piano was chosen to render sound outputs. With its standard 88 keys, the piano offers a broad pitch range. The piano's capability to play simultaneously across multiple octaves allows for the creation of complex harmonies. Note dynamics, affecting the loudness and forming a crucial element in musical expression, are mapped to the saturation values extracted from the image. As a representation of colour intensity, saturation values serve as a key parameter in shaping the dynamics of the musical output. The algorithm converts these saturation values into MIDI velocity values, influencing the strength and intensity of each note during MIDI playback. The calculation of note duration in the algorithm is based on the assumption that consecutive pixels on the same horizontal plane, with the same MIDI note value and identical velocity values, are treated as a single musical note, effectively representing a sustained occurrence. For example, if two consecutive eighth notes share the same MIDI note value and velocity value, they will be considered as a unified quarter note in terms of duration. While scrutinising the outputs obtained from processed images, structures akin to elements found in a musical composition were sought. These elements include components such as bass lines, melodic lines, chords, and arpeggios. Clusters of structures identified in the outputs have been examined, and recommendations are provided to unveil potential materials. It has been observed that although the quantities of these elements vary across different outputs, they are consistently present, and their representations are illustrated. In addressing the second hypothesis, which posits that the musical outputs obtained from digital images are suitable for presentation as music after composer intervention, the focus shifts towards the refinement of these outputs. In order to facilitate a more perceptible differentiation of the materials within the outputs and to ascertain their usability in a musical context, it was deemed necessary to present them in a manner distinct from the homogenous sound of the piano. Subsequently, an orchestration process is implemented, aiming to present the materials in a way that can be more easily discerned by the ear and be utilised in musical compositions. The orchestration process assumes a role in elevating the overall quality of the musical materials without the adding any extra notes. This stage is essential as it focuses on enhancing the raw musical output in terms of tonal balance, dynamics, and overall coherence. Importantly, the orchestration approach is carefully crafted to keep the fundamental essence of the original output intact. The objective is not to dilute or alter the core of these materials but rather to augment and enrich them in a manner that aligns with established principles of musicality. This preservation of the algorithmically generated core ensures that the intrinsic qualities of the digital imagederived musical elements persist, maintaining their authenticity amid the refining process. The second hypothesis asserts that, following intervention by a composer, the musical outputs derived from digital images are poised for presentation as music. The orchestration process becomes instrumental in realising this potential by refining the outputs and enhancing their expressive qualities for a more compelling musical presentation. This affirmation is responded to based on suppositions in the thesis, where the outcomes of the orchestration process not only responds to the hypothesis but also showcase the successful conversion of generated musical materials into potentially coherent compositions under the guidance of a composer's intervention The findings of this study serve as a foundational source for future research in the domain of computer-generated music and creative processes within music composition. The method demonstrated in this study not only showcases its current application but also lays the groundwork for the exploration of innovative techniques and tools in subsequent research. A crucial suggestion emanating from this study is the imperative to cultivate interdisciplinary collaboration among fields such as computer science, visual arts, psychology, music, and education sciences. Through collaboration, novel methods may be discovered, leading to a greater awareness of the synthesis of visual and auditory elements. Moreover, the study posits potential applications of its results in various domains, including live music performances, realtime music or image generation, and the integration of visual and auditory experiences. The impact of the developed method extends beyond the confines of the research study, indicating its versatility and potential to inspire further exploration in the realm of computer-generated music. Researchers and practitioners are strongly encouraged to push the boundaries of experimentation by manipulating different parameters and variables, thus revealing unique possibilities for artistic expression. The implications of the study extend into broader fields such as digital art and music, suggesting that its findings may significantly contribute to the evolving landscape of creative practises. The transformative nature of the method holds promise for influencing the utilisation of digital images as rich sources of musical inspiration. In essence, the implications and future directions elucidated by this study serve to position it not solely as an isolated exploration, but rather as a pivotal catalyst for subsequent research, collaborative initiatives, and advancements at the convergence of technology, art, and music.
Benzer Tezler
- Müziğin reklam filmlerinde kullanımı [Film (VHS video kaset)]
The use of music in commercials
AYŞE PELİN COŞKUN AKALIN
- Sinema filminde müziğin dramaturjiye yaptığı yapısal ve işlevsel katkı bağlamında film analizi
The analysis of the film in the context of the structural and functional contribution of the music in the cinema film to dramaturgy
ERDAL GÜNEY
- Image to music: Cross-modal melody generation through image captioning
İmajdan müziğe: İmaj altyazısı ile çapraz modlu melodi üretimi
ALPER KAPLAN
Yüksek Lisans
İngilizce
2022
Bilgisayar Mühendisliği Bilimleri-Bilgisayar ve KontrolYeditepe ÜniversitesiBilişsel Bilim Ana Bilim Dalı
DOÇ. DR. DIONYSIS GOULARAS
- Görüntüye eklenen müziğin görsel mesajların iletilmesine katkısı
The contribution of image based music to the transmission of visual messages
AYLİN ERBERİK BUDAK
Yüksek Lisans
Türkçe
1996
Güzel SanatlarAnkara ÜniversitesiGüzel Sanatlar Eğitimi Ana Bilim Dalı
PROF.DR. İNCİ SAN