I'm a Computer Music Phd student at UC San Diego. I love working with audio and sound. You can listen to my music here. I am currently teaching Music 173 at UCSD, a music production and composition class, and the website is here. The course website for the Fall 2018 iteration of that course is here. My old website from my years at CCRMA is here.
Jennifer Hsu
jsh008@ucsd.edu
My most recent research involves synthesizing percussive sounds. I am using modal synthesis and feedback frequency modulation (FM) techniques to synthesize sounds that are reminiscent of nonlinear percussion instruments. The technical portion of this project is described in a paper that I presented at the 2019 Sound and Music Computing (SMC) conference and can be found here. Accompanying sound examples can be found here. My PhD dissertation, titled "Physically-informed Percussion Synthesis with Nonlinearities for Real-time Applications", along with accompanying materials can be accessed here.
In learning about previous percussion synthesis methods, I implemented percussion models using the 2D waveguide mesh and finite difference approximations. The sounds that I synthesized through my experimentation are included here. Also, the gif above shows a 2D waveguide mesh in action after being excited by a 2D raised sinusoid!
The two following examples are 2D mesh waveguides
with 21x21 junctions, excited by an impulse, with
time-varying allpss filters at the boundaries and
convolved with (a) a noiseburst and (b) a room impulse
response.
To illustrate the impact of size on the mesh
waveguide synthesis,
the next two examples follow the same conditions as
above except the number of junctions has been changed
to 51x51.
These next four examples are finite difference
approximations for the von Karman plate model. The
parameters are set according to the material
properties of a steel plate. The finite difference
scheme used in these examples is from Stefan Bilbao's paper
"Sound Synthesis for Nonlinear Plates" (2005). In the first two examples,
the plate is given linear conditions with an inital
velocity of 20 m/s and 70 m/s, respectively. In the
last two examples, the plate is given nonlinear
conditions with 20 m/s and 70 m/s initial
velocity. Notice that under linear conditions, the two
examples sound quite similar except for amplitude
differences. Under nonlinear conditions, more
frequency components as well as a slight pitch glide appear in the 70 m/s initial
velocity example.
Using a real-time, modal synthesis technique, I
created two steel plate examples that have been
convolved with a noise burst and a room impulse response.
The RBM Drum Sequencer is implemented as a Pd patch and is capable of generating variations of an input beat pattern. The pattern variation uses a restricted Boltzmann machine (RBM) trained on Drum and Bass rhythms (or Drum and Bass + Brazillian-style beats). This system can generate rhythmic variations for patterns that have 3 instruments and intensity of each note is included in the algorithm.
Here is an example of the audio output. You
will first hear 4 bars of the basic, input beat. The
following bars are variations created with the RBM
trained on a Drum n' Bass dataset as well as the repeat module in the drum sequencer. Feel free to download the Pd
extern, patch, and Python RBM training code below to explore this tool in
real-time.
This project is a drum machine that is able to generate rhythmic variations of an input beat sequence. The drum machine is implemented as a Pure Data (Pd) patch. The rhythmic pattern variation is based on a canonical genetic algorithm that is implemented as a Pd extern object in c. While the generated rhythmic variations are interesting and seem to fit the input beat sequence, incorporating different output levels for the generated percussion beats would greatly improve the "natural-ness" of the generated variations.
Here is an example of the audio output. You
will first hear one bar of the basic, input beat. The
following bars are variations created with the genetic
algorithm as well as other modules in the drum machine
(mutes, circular shifts, and individual sample levels). Feel free to download the Pd
extern and patch below to explore this tool in
real-time.
The Ghost Note VST is a MIDI Effect plugin (written with JUCE and C++) that creates variations in percussive rhythmic patterns. The user is able generate ghost notes at the 2nd, 3rd, and 4th 16th notes of a quarter note within a beat pattern by specifying probabilities for the appearance of those notes as well as the volume of those notes. The beats that appear are a repetition of the note that falls on the quarter note beat (on the 1st 16th note of the quarter note). The user is also able to remove notes from the original pattern by specifying the percentage of notes that should be removed.
Here are some examples of the audio output. You
will first hear one bar of the basic, input beat. The
following bars are variations created with the Ghost
Note VST. The last example is my favorite, so feel
free to skip the first two if you're short on time!
During the summer of 2015, I had an amazing experience as a reseach intern in the Media Technology Lab at Gracenote. I developed a method for estimating the time of the downbeat in music. When you count the beats in 4/4 music as "1 2 3 4 1 2 3 4 ..." or 3/4 music as "1 2 3 1 2 3 ...", the downbeat is the "1". My task was to find the time at which the "1" occurs in different audio signals. I used a machine learning approach instead of a strictly DSP approach for this task. I trained my models only on the Ballroom Dataset and tested them on other songs that were not in the dataset. This work could not have been done without the help of my supervisor at Gracenote, Greg Tronel.
Here are some examples of the audio output. You will hear the test audio with a metronome. The bleep with the high frequency is the estimated downbeat. I am only posting good examples here, so if you would like to hear some bad examples, feel free to contact me:
This is a collaborative project between Cheng-i Wang, Shlomo Dubnov, and myself. Using a Factor Oracle-like data structure, the Variable Markov Oracle, we search for repeated patterns in polyphonic music. This method is capable of discovering patterns in both audio and symbolic representations.
This project is for integrating a physical model of the voice with a live saxophone player. I implemented a physical model of the vocal tract and vocal folds that can morph between vowel sounds. I also developed a technique to specify the sounding frequency of the voice synthesis. Frequency trajectories extracted from the acoustic saxophone signal are used to control the pitch of the voice. Future work involves mapping other sonic aspects of the saxophone sound to voice model control parameters.
This is a multiband delay VST plugin. MBDelay allows you to split a sound into three frequency bands. The amplitude, feedback, and delay time for each band can be controlled separately. I am working on getting this to work on Mac OSX 10.10, but I have been experiencing issues with Apple vecLib FFT functions since updating my OS. If you have any suggestions, please send them my way.
This is a granular synthesis VST plugin. With GrannyGrains, you can chunk up your audio into grains (large or small) and play them either sequentially or in a shuffled manner. You can control the binaural spread of the grains, the feedback, and percentage of reversed grains. There is also an option to quantize the timing of the grains to the beat of your DAW.
Here's an example of the audio
output. The first few seconds are of a
clean music box signal. I manipulate
parameters on GrannyGrains for the rest
of the sound file.
This was an experiment to see if you
can create interesting music using a new
technique of concatenative synthesis. A user sorts small 10-second snippets
of audio ('source') into groups. The groups are
purely user-defined. The system uses
decision trees to understand the
structure of the grouping. The user
inputs a 'target' audio for what they
want the final output audio to sound
like. The system then takes the small
snippets of source audio that sound the
most 'similar' to each chunk of the
target audio, concatenates them
together, and outputs the sound.
Here are some examples of the audio output:
Contact me at jsh008@ucsd.edu