Signal Generators: Bandwidth-Enhanced Additive Resynthesis, Sound Morphing

lorisread, lorisplay, lorismorph

      lorisread    ktimpnt, ifilcod, istoreidx, kfreqenv, kampenv, kbwenv[, ifadetime]
  ar  lorisplay    ireadidx, kfreqenv, kampenv, kbwenv
      lorismorph   isrcidx, itgtidx, istoreidx, kfreqmorphenv, kampmorphenv, kbwmorphenv

Description

The Loris unit generators, lorisread, lorisplay, and lorismorph, perform Bandwidth-Enhanced additive reconstruction of a sound specified by a set of partials obtained from a Reassigned Bandwidth-Enhanced Analysis performed using the Loris software.

lorisread imports a set of bandwidth-enhanced partials from a SDIF-format data file, applying control-rate frequency, amplitude, and bandwidth scaling envelopes, and stores the modified partials in memory.

lorismorph morphs two stored sets of bandwidth-enhanced partials and stores a new set of partials representing the morphed sound. The morph is performed by linearly interpolating the parameter envelopes (frequency, amplitude, and bandwidth, or noisiness) of the bandwidth-enhanced partials according to control-rate frequency, amplitude, and bandwidth morphing functions.

lorisplay renders a stored set of bandwidth-enhanced partials using the method of Bandwidth-Enhanced Additive Synthesis implemented in the Loris software, applying control-rate frequency, amplitude, and bandwidth scaling envelopes.

For more information about sound morphing and manipulation using Loris and the Reassigned Bandwidth-Enhanced Additive Model, visit the Loris web site at www.cerlsoundgroup.org/Loris/

Initialization

ifilcod – integer or character-string denoting a control-file derived from reassigned bandwidth-enhanced analysis of an audio signal. An integer denotes the suffix of a file loris.sdif (e.g. loris.sdif.1); a character-string (in double quotes) gives a filename, optionally a full pathname. If not fullpath, the file is sought first in the current directory, then in the one given by the environment variable SADIR (if defined). The reassigned bandwidth-enhanced data file contains breakpoint frequency, amplitude, noisiness, and phase envelope values organized for bandwidth-enhanced additive resynthesis. The control data must conform to one of the SDIF formats that can be interpreted using the Loris library for sound modeling and manipulation. Memory usage depends on the size of the files involved, which are read and held entirely in memory during computation but are shared by multiple calls.

Loris stores partials in SDIF RBEP frames. Each RBEP frame contains one RBEP matrix, and each row in a RBEP matrix describes one breakpoint in a Loris partial. A RBEL frame containing one RBEL matrix describing the labeling of the partials may precede the first RBEP frame in the SDIF file. The RBEP and RBEL frame and matrix definitions are included in the SDIF file's header. In addition to RBEP frames, Loris can also read and write SDIF 1TRC frames. Since 1TRC frames do not represent bandwidth-enhancement or the exact timing of Loris breakpoints, their use is not recommended. 1TRC capabilities are provided to allow interchange with programs that are unable to handle RBEP frames.

istoreidx, ireadidx, isrcidx, itgtidx are labels that identify a stored set of bandwidth-enhanced partials. lorisread imports partials from a SDIF file and stores them with the integer label istoreidx. lorismorph morphs sets of partials labeled isrcidx and itgtidx, and stores the resulting partials with the integer label istoreidx. lorisplay renders the partials stored with the label ireadidx. The labels are used only at initialization time, and may be reused without any cost or benefit in efficiency, and without introducing any interaction between instruments or instances.

ifadetime (optional) – In general, partials exported from Loris begin and end at non-zero amplitude. In order to prevent artifacts, it is very often necessary to fade the partials in and out, instead of turning them abruptly on and off. Specification of a non-zero ifadetime causes partials to fade in at their onsets and to fade out at their terminations. This is achieved by adding two more breakpoints to each partial: one ifadetime seconds before the start time and another ifadetime seconds after the end time. (However, no breakpoint will be introduced at a time less than zero. If necessary, the onset fade time will be shortened.) The additional breakpoints at the partial onset and termination will have the same frequency and bandwidth as the first and last breakpoints in the partial, respectively, but their amplitudes will be zero. The phase of the new breakpoints will be extrapolated to preserve phase correctness. If no value is specified, ifadetime defaults to zero. Note that the fadetime may not be exact, since the partial parameter envelopes are sampled at the control rate (krate) and indexed by ktimpnt (see below), and not by real time.

Performance

lorisread reads pre-computed Reassigned Bandwidth-Enhanced analysis data from a file stored in SDIF format, as described above. The passage of time through this file is specified by ktimpnt, which represents the time in seconds. ktimpnt must always be positive, but can move forwards or backwards in time, be stationary or discontinuous, as a pointer into the analysis file. kfreqenv is a control-rate transposition factor: a value of 1 incurs no transposition, 1.5 transposes up a perfect fifth, and .5 down an octave. kampenv is a control-rate scale factor that is applied to all partial amplitude envelopes. kbwenv is a control-rate scale factor that is applied to all partial bandwidth or noisiness envelopes. The bandwidth-enhanced partial data is stored in memory with a specified label for future access by another generator.

lorismorph generates a set of bandwidth-enhanced partials by morphing two stored sets of partials, the source and target partials, which may have been imported using lorisread, or generated by another unit generator, including another instance of lorismorph. The morph is performed by interpolating the parameters of corresponding (labeled) partials in the two source sounds (for a full explanation, visit the Loris web site at www.cerlsoundgroup.org/Loris/). The sound morph is described by three control-rate morphing envelopes. kfreqmorphenv describes the interpolation of partial frequency values in the two source sounds. When kfreqmorphenv is 0, partial frequencies are obtained from the partials stored at isrcidx. When kfreqmorphenv is 1, partial frequencies are obtained from the partials at itgtidx. When kfreqmorphenv is between 0 and 1, the partial frequencies are interpolated between corresponding source and target partials. Interpolation of partial amplitudes and bandwidth (noisiness) coefficients are similarly described by kampmorphenv and kbwmorphenv.

lorisplay implements signal reconstruction using Bandwidth-Enhanced Additive Synthesis. The control data is obtained from a stored set of bandwidth-enhanced partials imported from an SDIF file using lorisread or constructed by another unit generator such as lorismorph. kfreqenv is a control-rate transposition factor: a value of 1 incurs no transposition, 1.5 transposes up a perfect fifth, and .5 down an octave. kampenv is a control-rate scale factor that is applied to all partial amplitude envelopes. kbwenv is a control-rate scale factor that is applied to all partial bandwidth or noisiness envelopes. The bandwidth-enhanced partial data is stored in memory with a specified label for future access by another generator.

This implementation of the Loris unit generators was written by Kelly Fitz. It is patterned after a prototype implementation of the lorisplay unit generator written by Corbin Champion, and based on the method of Bandwidth-Enhanced Additive Synthesis and on the sound morphing algorithms implemented in the Loris library for sound modeling and manipulation. For more information about Bandwidth-Enhanced Additive Synthesis, Reassigned Bandwidth-Enhanced Additive Model, and Loris, visit the Loris web site at www.cerlsoundgroup.org/Loris/

Examples

Example 1

; 
; Play the partials in clarinet.sdif 
; from 0 to 3 sec with 1 ms fadetime 
; and no frequency , amplitude, or 
; bandwidth modification.
;
instr 1
    ktime    linseg      0, p3, 3.0    ; linear time function from 0 to 3 seconds
             lorisread   ktime, "clarinet.sdif", 1, 1, 1, 1, .001
    asig     lorisplay   1, 1, 1, 1
             out         asig
endin

Example 2

; 
; Play the partials in clarinet.sdif 
; from 0 to 3 sec with 1 ms fadetime 
; adding tuning and vibrato, increasing the 
; "breathiness" (noisiness) and overall
; amplitude, and adding a highpass filter.
;
instr 2
    ktime    linseg      0, p3, 3.0    ; linear time function from 0 to 3 seconds
    
    ; compute frequency scale for tuning
    ; (original pitch was G#4)
    ifscale  =           cpspch(p4)/cpspch(8.08)

    ; make a vibrato envelope
    kvenv    linseg      0, p3/6, 0, p3/6, .02, p3/3, .02, p3/6, 0, p3/6, 0
    kvib     oscil       kvenv, 4, 1   ; table 1, sinusoid

    kbwenv   linseg      1, p3/6, 1, p3/6, 2, 2*p3/3, 2
             lorisread   ktime, "clarinet.sdif", 1, 1, 1, 1, .001
    a1       lorisplay   1, ifscale+kvib, 2, kbwenv  
    a2       atone       a1, 1000      ; highpass filter, cutoff 1000 Hz
             out         a2
endin

The instrument in Example 1 synthesizes a clarinet tone from beginning to end using partials derived from reassigned bandwidth-enhanced analysis of a three-second clarinet tone, stored in a file, "clarinet.sdif." The instrument in Example 2 adds tuning and vibrato to the clarinet tone synthesized by instr 1, boosts its amplitde and noisiness, and applies a highpass filter to the result. The following score can be used to test both of the instruments described above.

	; make sinusoid in table 1
	f 1 0 4096 10 1
	
	; play instr 1
	;     strt   dur
	i 1    0      3
	i 1    +      1
	i 1    +      6
	s
	
	; play instr 2
	;     strt   dur   ptch
	i 2     1     3    8.08
	i 2     3.5   1    8.04
	i 2     4     6    8.00
	i 2     4     6    8.07

    e

Morphing Example 1

;
;
; Morph the partials in clarinet.sdif into the
; partials in flute.sdif over the duration of
; the sustained portion of the two tones (from
; .2 to 2.0 seconds in the clarinet, and from
; .5 to 2.1 seconds in the flute). The onset
; and decay portions in the morphed sound are
; specified by parameters p4 and p5, respectively.
; The morphing time is the time between the
; onset and the decay. The clarinet partials are
; shfited in pitch to match the pitch of the flute
; tone (D above middle C). 
;
;
instr 1
    ionset   =           p4
    idecay   =           p5
    itmorph  =           p3 - (ionset + idecay)
    ipshift  =           cpspch(8.02)/cpspch(8.08)
    
    ktcl     linseg      0, ionset, .2, itmorph, 2.0, idecay, 2.1    ; clarinet time function, morph from .2 to 2.0 seconds
    ktfl     linseg      0, ionset, .5, itmorph, 2.1, idecay, 2.3    ; flute time function, morph from .5 to 2.1 seconds
    kmurph   linseg      0, ionset, 0, itmorph, 1, idecay, 1
             lorisread   ktcl, "clarinet.sdif", 1, ipshift, 2, 1, .001
             lorisread   ktfl, "flute.sdif", 2, 1, 1, 1, .001
             lorismorph  1, 2, 3, kmurph, kmurph, kmurph
    asig     lorisplay   3, 1, 1, 1
             out         asig
endin

Morphing Example 2

;
; Morph the partials in trombone.sdif into the
; partials in meow.sdif. The start and end times
; for the morph are specified by parameters p4 
; and p5, respectively. The morph occurs over the
; second of four pitches in each of the sounds, 
; from .75 to 1.2 seconds in the flutter-tongued
; trombone tone, and from 1.7 to 2.2 seconds in 
; the cat's meow. Different morphing functions are
; used for the frequency and amplitude envelopes,
; so that the partial amplitudes make a faster  
; transition from trombone to cat than the frequencies. 
; (The bandwidth envelopes use the same morphing 
; function as the amplitudes.) 
;
instr 2
    ionset   =           p4
    imorph   =           p5 - p4
    irelease =           p3 - p5
        
    kttbn    linseg      0, ionset, .75, imorph, 1.2, irelease, 2.4
    ktmeow   linseg      0, ionset, 1.7, imorph, 2.2, irelease, 3.4
    
    kmfreq   linseg      0, ionset, 0, .75*imorph, .25, .25*imorph, 1, irelease, 1
    kmamp    linseg      0, ionset, 0, .75*imorph, .9, .25*imorph, 1, irelease, 1
    
             lorisread   kttbn, "trombone.sdif", 1, 1, 1, 1, .001
             lorisread   ktmeow, "meow.sdif", 2, 1, 1, 1, .001
             lorismorph  1, 2, 3, kmfreq, kmamp, kmamp
    asig     lorisplay   3, 1, 1, 1
             out         asig
endin

The instrument in Morphing Example 1 performs a sound morph between a clarinet tone and a flute tone using reassigned bandwidth-enhanced partials stored in "clarinet.sdif" and "flute.sdif."

The morph is performed over the sustain portions of the tones, 2. seconds to 2.0 seconds in the case of the clarinet tone and .5 seconds to 2.1 seconds in the case of the flute tone. The time index functions, ktcl and ktfl, align the onset and decay portions of the tones with the specified onset and decay times for the morphed sound, specified by parameters p4 and p5, respectively. The onset in the morphed sounds is purely clarinet partial data, and the decay is purely flute data. The clarinet partials are shifted in pitch to match the pitch of the flute tone (D above middle C).

The instrument in Morphing Example 2 performs a sound morph between a flutter-tongued trombone tone and a cat's meow using reassigned bandwidth-enhanced partials stored in "trombone.sdif" and "meow.sdif." The data in these SDIF files have been channelized and distilled to establish correspondences between partials.

The two sets of partials are imported and stored in memory locations labeled 1 and 2, respectively. Both of the original sounds have four notes, and the morph is performed over the second note in each sound (from .75 to 1.2 seconds in the flutter-tongued trombone tone, and from 1.7 to 2.2 seconds in the cat's meow). The different time index functions, kttbn and ktmeow, align those segments of the source and target partial sets with the specified morph start, morph end, and overall duration parameters. Two different morphing functions are used, so that the partial ammplitudes and bandwidth coefficients morph quickly from the trombone values to the cat's-meow values, and the frequencies make a more gradual transition. The morphed partials are stored in a memory location labeled 3 and rendered by the subsequent lorisplay instruction. They could also have been used as a source for another morph in a three-way morphing instrument. The following score can be used to test both of the instruments described above.

	; play instr 1
	;     strt   dur   onset   decay
	i 1    0      3     .25     .15
	i 1    +      1     .10     .10
	i 1    +      6    1.      1.
	s
	
	; play instr 2
	;     strt   dur  morph_start   morph_end
	i 2    0      4     .75           2.75
	
	e

Author

Kelly Fitz
WSU, Pullman
2002
email:loris@cerlsoundgroup.org

Last update: 11 June 2002


Signal Generators: Bandwidth-Enhanced Additive Resynthesis, Sound Morphing