DeepJams is a graduate project at UC Berkeley exploring ways to augment traditional music composition with machine intelligence. We are applying the latest research in the field along with the latest machine learning toolkits and artificial neural networks to train models that can extend original human compositions with equally original machine generated extensions.
Our group's focus centers on creating useable models which exceed not only on quantitative dataset benchmarking exercises, but can also garner positive qualitative feedback via controlled user testing. To this end, we have created fully functional prototypes for every iteration of the academic effort, free for evaluation and use.
We envision this not only as an academic project (which we hope to publish) but also as a functional product with real use-cases and potentially licensable intellectual property.
1
Jamming out solo on guitar or piano, and stuck or bored
Its hard to interact with a machine learning model with a guitar in your hand, so we made it as
easy as uploading a file. Check out the snippet of one of the team members picking on a guitar, and
the corresponding output.
Input:
Output:
2
Learning about music theory when composing songs
This was not a use case initially imagined, but was brought to our attention from our more
musically inclined friends. To further help composers, we have incorporated sheet music to go along
with the generated midi output. We have heard some really fun feedback on how folks are looking at
chord progressions and melodies! Here is an example of generating the next 30 seconds on a classical
piano piece.
Input:
Output:
3
Driving engagement to music apps and websites
Imagine listening to a song, now imagine endless variations and remixes. Check out what it can
sound like for something like Sandstorm. Imagine possibly even playing both snippets at the same time…
or off beat :).
Input:
Output:
You can start using our application here. Just follow the two easy steps below or see the usage demo video on the right.
Step 1) upload an MP3 file, much like this one (we all happen to be huge fans of Sandstorm)
Under the hood, we take care of all conversions, audio pipelines, and threshold cleanup.
Step 2) listen to what the next few bars of your music can sound like
Music is one of the most universal forms of expression. It can be pleasing, tear-jerking, or motivating. It accompanies us on journeys, at parties, at home, at work, and keeps us company in our solitude. It conjures memories, supports traditions, and sometimes even generates stereotypes. It is so woven into our lives, sometimes we don’t even notice it. All three of us are passionate about music, passionate about science, and even more passionate about furthering the application of science to music.
Composing music is challenging. Composers want to create great melodies, but each new piece needs to be unique. Very often, composers try to step outside their comfort zone (what they have done before), in order to create something “new”. There is a fine balance between being predictable in your sounds, but also surprising the listener.
With DeepJams, we are not creating from scratch. We are creating based on your input, allowing you to keep your signature and style. We focused on intelligent tooling for composers using state-of-the-art machine learning science. All our tools are available for immediate use, and we love to receive feedback.
Byron is a Data Scientist at Pandora, working with and (primarily) learning from some of the smartest and most passionate music lovers. Favorite activity is going through all of my thumbed up songs on Pandora, and jamming out on my guitar or piano. Would love more jam sessions, but it’s not always easy finding people to play with, thus this project :). Most importantly, the biggest Taylor Swift fan on the team.
Andrea is a freelance data scientist with close to twenty years of consulting experience. Andrea studied music through college while pursuing her engineering degree. Combining music with machine learning brings all of her loves together. Andrea likes music best when it is live, and tries to catch shows as often as possible. From classical to classic rock, blues to pop, and jam bands to musicals (Lin Manuel Miranda is amazing!), if you are with Andrea, music is most likely playing.
Saif is a former Wall Street quant with fifteen years at hedge funds and three as CTO of a AI-driven-diagnostic startup he co-founded. Before all this, Saif produced a rap EP back in high school, focusing on electronic beats before there was any real tooling (1995!) Merging his musical obsessions with professional engineering, data science, and machine learning interests is a dream come true. This stuff shouldn't even count as homework!
This project is being conducted at the School of Information at UC Berkeley and being advised by Professor Joyce Shen and Professor Alberto Todeschini. We greatly appreciate their constant guidance and feedback through this journey. We thank Dr. D. Alex Hughes from UC Berkeley for his guidance on experimental validation.
We thank Startup@BerkeleyLaw and The Berkeley Center for Law, Business and the Economy and UC Berkeley Intellectual Property and Industry Research Alliances (IPIRA) for their time and guidance on how to carry the project forward to licensable state.
We thank Amazon Web Services for providing generous usage credits which allowed us to lease compute resources for our application.
Our work is built extensively upon recent research, open source projects, and various academic efforts.
"In some ways, music is a lot like literature. It can be pleasing, enlightening, even life-changing. A great song, like a great book, affects the listener in unanticipated ways and resonates long after the music has ceased. And, if done well, both music and literature have a rhythm and tone unique to their authors; you wouldn't confuse a passage from a Charles Dickens novel with one from a Harry Potter book, nor would you mistake a verse from A Hard Day's Night for one from a Lady Gaga song." --Leah Butterfield, 2014, www.bustle.com
Our project took great pains to quantitatively and qualitatively measure the originality of our model outputs. There is a fine balance -- the outputs need to extend the gist of the user's input, but also produce original music reflecting the style of the training corpora (without being too close or too different.) We trained custom models on sub-genres of music.
We are aiming to publish a paper with quantitative results. However, no need to wait. Please start using the application and tell us what you think!
We are currently not asserting any rights over the output, so you own your own outputs. Of course, please ensure you have the appropriate rights over the input content that you seed the process with. We are currently discussing with lawyers how far outputs must diverge before even the original content rights do not matter.
All music will be processed, but the system works well on classical, and particularly on piano music. Generally, discrete inputs produce better outputs than ambient pieces.
Yes, for now all inputs and outputs are stored to help debug and improve the system.
Please use the system and provide critical feedback. Introductions to musical artists of any level would also be much appreciated.
We have built everything with a stateless REST architecture, but currently only have a web facade. We're exploring more extemporaneous facades better suited for casual musicians -- a smartphone app and an Alexa Skill. Let us know if you can help beta test.
We are pretty scalable but we're in the midst of architecting for infinite scalability via serverless and lambda architectures to handle our entire execution pipeline.
No, that only happens in the movies. In reality, we spent dozens of nights and weekends just to get the system functional. We'd be pretty happy if it could just start doing stuff without effort. Until then, this is human labour of love.
Last updated: December 13, 2017
Please read these Terms of Service ("Terms", "Terms of Service") carefully before using the www.deepjams.com website (the "Service") operated by DeepJams ("us", "we", or "our").
Your access to and use of the Service is conditioned on your acceptance of and compliance with these Terms. These Terms apply to all visitors, users and others who access or use the Service.
By accessing or using the Service you agree to be bound by these Terms. If you disagree with any part of the terms then you may not access the Service. Terms of Service from TermsFeed for DeepJams.
Our Service may contain links to third-party web sites or services that are not owned or controlled by DeepJams.
DeepJams has no control over, and assumes no responsibility for, the content, privacy policies, or practices of any third party web sites or services. You further acknowledge and agree that DeepJams shall not be responsible or liable, directly or indirectly, for any damage or loss caused or alleged to be caused by or in connection with use of or reliance on any such content, goods or services available on or through any such web sites or services.
We strongly advise you to read the terms and conditions and privacy policies of any third-party web sites or services that you visit.
We may terminate or suspend access to our Service immediately, without prior notice or liability, for any reason whatsoever, including without limitation if you breach the Terms.
All provisions of the Terms which by their nature should survive termination shall survive termination, including, without limitation, ownership provisions, warranty disclaimers, indemnity and limitations of liability.
These Terms shall be governed and construed in accordance with the laws of California, United States, without regard to its conflict of law provisions.
Our failure to enforce any right or provision of these Terms will not be considered a waiver of those rights. If any provision of these Terms is held to be invalid or unenforceable by a court, the remaining provisions of these Terms will remain in effect. These Terms constitute the entire agreement between us regarding our Service, and supersede and replace any prior agreements we might have between us regarding the Service.
We reserve the right, at our sole discretion, to modify or replace these Terms at any time. If a revision is material we will try to provide at least 15 days notice prior to any new terms taking effect. What constitutes a material change will be determined at our sole discretion.
By continuing to access or use our Service after those revisions become effective, you agree to be bound by the revised terms. If you do not agree to the new terms, please stop using the Service.
If you have any questions about these Terms, please contact us.
Last updated: December 13, 2017
DeepJams ("us", "we", or "our") operates the www.deepjams.com website (the "Service").
This page informs you of our policies regarding the collection, use and disclosure of Personal Information when you use our Service.
We will not use or share your information with anyone except as described in this Privacy Policy. Privacy Policy from TermsFeed for DeepJams.
We use your Personal Information for providing and improving the Service. By using the Service, you agree to the collection and use of information in accordance with this policy. Unless otherwise defined in this Privacy Policy, terms used in this Privacy Policy have the same meanings as in our Terms and Conditions, accessible at www.deepjams.com
While using our Service, we may ask you to provide us with certain personally identifiable information that can be used to contact or identify you.
We collect information that your browser sends whenever you visit our Service ("Log Data"). This Log Data may include information such as your computer's Internet Protocol ("IP") address, browser type, browser version, the pages of our Service that you visit, the time and date of your visit, the time spent on those pages and other statistics.
Cookies are files with small amount of data, which may include an anonymous unique identifier. Cookies are sent to your browser from a web site and stored on your computer's hard drive.
We use "cookies" to collect information. You can instruct your browser to refuse all cookies or to indicate when a cookie is being sent. However, if you do not accept cookies, you may not be able to use some portions of our Service.
We may employ third party companies and individuals to facilitate our Service, to provide the Service on our behalf, to perform Service-related services or to assist us in analyzing how our Service is used.
These third parties have access to your Personal Information only to perform these tasks on our behalf and are obligated not to disclose or use it for any other purpose.
The security of your Personal Information is important to us, but remember that no method of transmission over the Internet, or method of electronic storage is 100% secure. While we strive to use commercially acceptable means to protect your Personal Information, we cannot guarantee its absolute security.
Our Service may contain links to other sites that are not operated by us. If you click on a third party link, you will be directed to that third party's site. We strongly advise you to review the Privacy Policy of every site you visit.
We have no control over, and assume no responsibility for the content, privacy policies or practices of any third party sites or services.
Our Service does not address anyone under the age of 13 ("Children").
We do not knowingly collect personally identifiable information from children under 13. If you are a parent or guardian and you are aware that your Children has provided us with Personal Information, please contact us. If we discover that a Children under 13 has provided us with Personal Information, we will delete such information from our servers immediately.
We may update our Privacy Policy from time to time. We will notify you of any changes by posting the new Privacy Policy on this page.
You are advised to review this Privacy Policy periodically for any changes. Changes to this Privacy Policy are effective when they are posted on this page.
If you have any questions about this Privacy Policy, please contact us.
Last updated: December 13, 2017
Please read these Terms and Conditions ("Terms", "Terms and Conditions") carefully before using the www.deepjams.com website (the "Service") operated by DeepJams ("us", "we", or "our").
Your access to and use of the Service is conditioned on your acceptance of and compliance with these Terms. These Terms apply to all visitors, users and others who access or use the Service.
By accessing or using the Service you agree to be bound by these Terms. If you disagree with any part of the terms then you may not access the Service. Terms and Conditions from TermsFeed for DeepJams.
Our Service may contain links to third-party web sites or services that are not owned or controlled by DeepJams.
DeepJams has no control over, and assumes no responsibility for, the content, privacy policies, or practices of any third party web sites or services. You further acknowledge and agree that DeepJams shall not be responsible or liable, directly or indirectly, for any damage or loss caused or alleged to be caused by or in connection with use of or reliance on any such content, goods or services available on or through any such web sites or services.
We strongly advise you to read the terms and conditions and privacy policies of any third-party web sites or services that you visit.
We may terminate or suspend access to our Service immediately, without prior notice or liability, for any reason whatsoever, including without limitation if you breach the Terms.
All provisions of the Terms which by their nature should survive termination shall survive termination, including, without limitation, ownership provisions, warranty disclaimers, indemnity and limitations of liability.
These Terms shall be governed and construed in accordance with the laws of California, United States, without regard to its conflict of law provisions.
Our failure to enforce any right or provision of these Terms will not be considered a waiver of those rights. If any provision of these Terms is held to be invalid or unenforceable by a court, the remaining provisions of these Terms will remain in effect. These Terms constitute the entire agreement between us regarding our Service, and supersede and replace any prior agreements we might have between us regarding the Service.
We reserve the right, at our sole discretion, to modify or replace these Terms at any time. If a revision is material we will try to provide at least 15 days notice prior to any new terms taking effect. What constitutes a material change will be determined at our sole discretion.
By continuing to access or use our Service after those revisions become effective, you agree to be bound by the revised terms. If you do not agree to the new terms, please stop using the Service.
If you have any questions about these Terms, please contact us.