1. NOTICE: The game is still in Early Access, which does not constitute a Launch! Please keep feedback and concerns tempered during this still-in-development period. We are working hard to make the game ready for Launch, based on your feedback.
    Dismiss Notice

Framework for playing instruments in game using MML or ABC notation

Discussion in 'Developer Depot' started by Eniko, Sep 23, 2014.

  1. Eniko

    Eniko Avatar

    Messages:
    115
    Likes Received:
    518
    Trophy Points:
    18
    Gender:
    Female
    EDIT: Since DropBox public folders no longer work, the midi player has been rehosted on Google Drive.

    Midi to MML/ABC converters:

    For MML there's:
    For ABC there's:
    Once you've converted a midi file to abc or mml, you can try the custom midi player to test it out outside the game. If your ABC song was made for LOTRO and does not sound correct, you can try adding the following text to the top of the file:

    Code:
    %%lotro-compatible
    You can also convert WAV to Midi using AmazingMIDI. (credit @redfish)
    ____________________________________________________

    TL;DR: this library securely plays ABC and MML songs, it is the only library which supports both, and in doing so supports the extensive music libraries made for Mabinogi, ArcheAge and Lord of the Rings Online, all MMOs which have had in-game instruments as a feature. It handles parsing, security and timing, and generates events for notes that need to be played. Implementation of audio is up to the end-user. It comes with a program which plays MML and ABC songs through midi, for out of game testing of songs by devs or musicians, as well as samples which show how to implement audio. I made this library free (MIT licensed) to give back to the community. Using this library would allow the game to support a vast library of pre-existing songs, significantly lowering the barrier to entry to playing instruments in game for players.
    ____________________________________________________

    Final edit: I consider this framework complete now, short of bug fixes or requests by the dev team for added functionality.
    Final tally is over 3,000 lines of code and an MIT licensed repository available for all. This first post is unlikely to change beyond this point.
    ____________________________________________________

    (Asked around and was told it was probably best to post here, as my submission would be code. Devs, if you are interested please let me know what the best way is to submit this! For now I've created a GitHub repository so you can look at the code. Also this post discusses audio implementation details as they pertain to SotA)
    ____________________________________________________

    So a few years ago I wrote a library that takes MML or ABC notation strings, parses them and "plays" them. First off, MML and ABC are two popular text notations for transcribing music into simple plain text, MML is used to play music on instruments in the MMOs Mabinogi and Archeage, ABC notation is used in general as well as in Lord of the Rings Online. All of these have online repositories of ready made songs transcribed into MML or ABC. I've been polishing it up so it can be used in Shroud.

    Use of this library would make it possible for players to play pretranscribed music in MML or ABC format inside the game, allowing bards to create and play songs.

    A video/song is worth a thousand words, so here's a sample program which plays Stones using this framework with midi by reading a Mabinogi MML file:



    You can try it out yourself by downloading the midi player from my public Google Drive. Windows only, requires .NET framework 4.5, small selection of songs is included, ABC playback is LotRO compatible. I heard it can take a few seconds to start up, though it's instant on my system.
    ____________________________________________________

    The library is made in C# and so should be Unity compatible. The main components are:
    • ABCPlayer: this loads and plays ABC format songs. Override PlayNote(Note note, int channel) to implement audio.
    • MultiTrackMMLPlayer: this loads and plays MML format songs with multiple tracks. To implement audio override the same method as above.
    • IMusicPlayer: an interface shared by MusicPlayer (the base class for ABCPlayer and MMLPlayer) as well as the MultiTrackMMLPlayer. Supports loading from file, string or stream, start/stop playback, seek, and mute/unmute.
    Notes are structs that have the following properties: Octave, Length, Type (the base note, abcdefg), Sharp, and Volume. Note structs also have a GetFrequency function which returns its numerical frequency relative to a tuning Note, or the A note in the 4th octave if unspecified.

    It is incredibly easy to use and easy to integrate into any game which can pass the current time as a TimeSpan to a music player's Update method. Keep in mind the framework only parses MML and ABC songs and takes care of timing, it does not generate sound; that's up to the program using the library.

    MML example:
    Code:
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using TextPlayer;
    using TextPlayer.MML;
    
    namespace MMLTest {
        public class BeepPlayer : MMLPlayer {
            public BeepPlayer()
                : base() {
            }
    
            protected override void PlayNote(Note note, int channel) {
                // length is halved because Console.Beep messes up using full length on consecutive notes
                Console.Beep((int)note.GetFrequency(), (int)(note.Length.TotalMilliseconds * 0.5));
            }
        }
    
        class Program {
            static void Main(string[] args) {
                BeepPlayer player = new BeepPlayer();
                player.FromFile("test.mml");
                player.Play();
    
                while (player.Playing) {
                    player.Update();
                }
            }
        }
    }
    
    ABC example:
    Code:
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using TextPlayer;
    using TextPlayer.ABC;
    
    namespace ABCTest {
        public class BeepPlayer : ABCPlayer {
            public BeepPlayer()
                : base(false) {
            }
    
            protected override void PlayNote(Note note, int channel) {
                // Only play melody (channel 0) and first note of chords (channel 1)
                if (channel <= 1) {
                    // length is halved because Console.Beep messes up using full length on consecutive notes
                    Console.Beep((int)note.GetFrequency(), (int)(note.Length.TotalMilliseconds * 0.5));
                }
            }
        }
    
        class Program {
            static void Main(string[] args) {
                BeepPlayer player = new BeepPlayer();
                player.FromFile("test.abc");
                player.Play();
    
                while (player.Playing) {
                    player.Update();
                }
            }
        }
    }
    Some other notes:
    • The ABC player was designed to be compatible with Lord of the Rings Online's implementation of ABC and its music archives, some advanced ABC features may not be implemented or work correctly.
    • The MML player was designed to be compatible with Mabinogi's MML implementation and none of the metadata associated with MML's wider, more traditional use is supported.
    • I've tested both ABC and MML pretty extensively, but MML is definitely the best tested format, the most feature complete and the most robust implementation. I personally also prefer MML due to its transcribing simplicity for those playing the game, ABC is way more complex and seems like overkill to me, at least for use in a game, and also much harder to parse and implement.
    • EDIT: sgt_pEppEr on IRC suggested that Unity uses .NET 2.0 so I just checked if it'll compile under an older profile. After changing some IsNullOrWhiteSpace() calls it compiled without a hitch.
    • EDIT2: After finding out ArcheAge uses a slightly different version of MML (tempo commands only apply to the track they're found in, volume range 1-127) I have added a Mode switch to the MML players, so both are properly supported.
     
    Last edited: Dec 29, 2016
  2. Mata

    Mata Avatar

    Messages:
    4,101
    Likes Received:
    10,090
    Trophy Points:
    153
    Gender:
    Female
    Location:
    Ruhrpott - Deutschland
    Welcome Eniko!
     
    Epona, Sir Cabirus, mbomber and 4 others like this.
  3. WrathPhoenix

    WrathPhoenix Avatar

    Messages:
    1,249
    Likes Received:
    4,396
    Trophy Points:
    113
    Our port phoenix citizens.... we put them right to work... *nods sagely*
     
  4. Eniko

    Eniko Avatar

    Messages:
    115
    Likes Received:
    518
    Trophy Points:
    18
    Gender:
    Female
    Gotta be productive. :D Thanks for the welcome!
     
    Hornpipe, Epona, Sir Cabirus and 8 others like this.
  5. smack

    smack Avatar

    Messages:
    6,734
    Likes Received:
    14,091
    Trophy Points:
    153
    @Eniko looks cool! But how would something like this work in-game between players? Let's say I did want to play Stones on my in-game piano. Is there some interface that allows me to configure the piano object to play that Stones file? And how will other players hear the song I'm playing? Is there code to send the MML stream to other players so they would hear it locally?

    And welcome to the community! :)
     
  6. Eniko

    Eniko Avatar

    Messages:
    115
    Likes Received:
    518
    Trophy Points:
    18
    Gender:
    Female
    The way Mabinogi did was you had a Composing skill. That skill allowed you to take music scrolls which had three fields (one field for each track or voice) and write the MML in it (and a button to import from clipboard). Depending on your skill levels in music you had a limited length of characters for every voice. Once transcribed if you had an instrument equipped you could play the entire scroll at once. I don't know how it works in LotRO or Archeage so maybe someone else can comment on those.

    So the easiest way is to let the player input the code into a musical scroll object, which when combined with an instrument will play the music, I think. The player would send the MML code to the server (MML files are quite compact, Stones is only 2033 bytes long) and that would send it to the clients, notifying them player X is now playing this song. By playing transcribed songs you get around a problem Guild Wars 2 has where it's all realtime, and your enjoyment of the song can vary greatly due to lag. Since the MML (or ABC) is all sent in one go, that's not an issue.

    Again I wanna stress I wrote this originally to allow barding within my own games, and decided to support MML and ABC because these are the two big in-game music formats with archives of songs all over the web already, making it easier for people new to the format to get into it. I still think though that ABC is at its heart way too complex, and it's probably less confusing to support only one or the other, in which case my vote strongly goes towards MML.
     
    Epona, Portia, Ice Queen and 8 others like this.
  7. Beaumaris

    Beaumaris Avatar

    Messages:
    3,229
    Likes Received:
    5,685
    Trophy Points:
    165
    Gender:
    Male
    Location:
    New Mexico
    Welcome to the forums :)

    Good opening post. Player-driven music systems are fun and I hope one makes it into SOTA.
     
  8. smack

    smack Avatar

    Messages:
    6,734
    Likes Received:
    14,091
    Trophy Points:
    153
    Thanks for the explanation, Eniko! One more question: if a group of players wanted to play a symphony, is there some mechanism to have them all start at the same time and stay in sync?
     
  9. Eniko

    Eniko Avatar

    Messages:
    115
    Likes Received:
    518
    Trophy Points:
    18
    Gender:
    Female
    In Mabinogi there was a special action that when used by the party leader would start everyone off playing their currently equipped instrument and music scroll. There's quite a few really good videos of how that works out in practice on YouTube, it can be pretty amazing and I always thought being a musician in Mabinogi was tons of fun. In fact, lemme link you some:

     
    mbomber, Portia, Ice Queen and 9 others like this.
  10. Baron Drocis Fondorlatos

    Baron Drocis Fondorlatos Avatar

    Messages:
    17,485
    Likes Received:
    33,663
    Trophy Points:
    153
    Gender:
    Male
    Very cool!

    Here's the question I have for you but probably also the devs. This is great and I hope SOTA has this functionality. But more so I'd like the ability to write songs that are scripted with integration into the music. So I can control when my character sings or reads a poem at a certain moment in time. To the OP, can your program do that? To the Devs, can we do that?
     
  11. Beli

    Beli Avatar

    Messages:
    956
    Likes Received:
    2,435
    Trophy Points:
    105
    Gender:
    Male
    Location:
    Alabama
    Welcome Eniko. I am still not sure how they will incorproate the players to write and play instruments in the game yet. I have been hoping for ABC. When I heard about Shroud, I was told that you will be able to play music in game. Its was one of the selling points for me.
     
  12. Beli

    Beli Avatar

    Messages:
    956
    Likes Received:
    2,435
    Trophy Points:
    105
    Gender:
    Male
    Location:
    Alabama
    Its doable. SWG had the same thing as LOTRO does.
     
  13. Eniko

    Eniko Avatar

    Messages:
    115
    Likes Received:
    518
    Trophy Points:
    18
    Gender:
    Female
    Neither MML nor ABC format comes with support for such things. There might be possibilities for adding it outside the format, but that's kind of out of the scope of my library, unfortunately. :(

    Thanks for the welcome! This library does support ABC, though I'm not sure how good my implementation is. Maybe I should make a stand alone player, for people to be able to test songs with so I can find and fix bugs? I never used ABC myself, so I may have missed something.
     
  14. Gubbles

    Gubbles Avatar

    Messages:
    856
    Likes Received:
    2,198
    Trophy Points:
    105
    Gender:
    Male
    Location:
    Corvus Peak
    Cool project Eniko. I hope this makes it into the game.
     
  15. Jynx

    Jynx Avatar

    Messages:
    721
    Likes Received:
    1,933
    Trophy Points:
    105
    Gender:
    Female
    Location:
    United Kingdom
    Could be a good use for the metronome that's already in game. Set it up to sync people playing in a group.
     
  16. DavenRock [MGT]

    DavenRock [MGT] Avatar

    Messages:
    884
    Likes Received:
    1,890
    Trophy Points:
    105
    Gender:
    Male
    Location:
    United States
    Lotro used a /play songhere.abc command to play a desired instrument for the extent of the content in the .abc file. Some files contained only drum patterns, some were made only for specific instruments. However, each .abc document was created by someone. The interface and everything they use in lotro towards .abc files feels really easy to use.
     
  17. Eniko

    Eniko Avatar

    Messages:
    115
    Likes Received:
    518
    Trophy Points:
    18
    Gender:
    Female
    Yeah, that may be a way easier way to implement it! The library already works to play Lotro style ABC songs like that, and I've done some more work to make the traditional multiple voice style (usually 3 tracks) of MML as seen in Archeage and Mabinogi work out of the box. That way the /play command could take song.abc or song.mml. I suppose I'll create a player that can take either and figure out which it is itself, next. ;)

    I'm also working on a stand-alone MML and ABC player for Windows, so people can try out songs and see what all the fuss is about. I already have the MML part of it working, just need to integrate ABC support and clean up the GUI a bit. :)
     
  18. Eniko

    Eniko Avatar

    Messages:
    115
    Likes Received:
    518
    Trophy Points:
    18
    Gender:
    Female
    I've created a stand-alone midi player for ABC and MML files. Try it out by downloading the midi player from my public DropBox. Windows, requires .NET framework 4.5, small selection of songs is included, ABC playback is LotRO compatible. I heard it can take a few seconds to start up, though it's instant on my system. I've also edited the link into the OP.

    I've tested MML pretty extensively and I think it's basically perfect, I can't speak for ABC so if you find LotRO compatible songs that don't play right, let me know where I can find the song and what time it goes wrong. If you want to find more songs, here are some extra search queries to find some. Keep in mind not all MML songs are properly formatted, MML files should start with 'MML@' and end with a semi-colon ';'.
     
  19. Pyro

    Pyro Avatar

    Messages:
    183
    Likes Received:
    157
    Trophy Points:
    28
    I've been a fan of bard skills since the kickstarter days. I definitely want to see some attention paid to making it enjoyable and potentially also useful in game (UO Bard here)

    Also on a side note, I love roleplaying a bard, but in all reality, I have next to zero musical skill whatsoever (it's why I enjoy the roleplay of it). So, if it's fun for me, and powerful enough for actual minstrels, amazing.

    Just hoping because I'm less musically talented than a fish flopping out of water.
     
  20. Eniko

    Eniko Avatar

    Messages:
    115
    Likes Received:
    518
    Trophy Points:
    18
    Gender:
    Female
    The good part is for the musically challenged there are programs that will convert midi to MML (the actual program is in English) and ABC so even if the wealth of MML and ABC files already on the web is insufficient, you can use those to easily create more even if you're not gifted yourself. :) Plus MML in particular is very easy to write, there's only about a dozen commands. There's a pretty good overview on this wiki.

    EDIT: Not to mention if this makes it into the game as a feature you can bet your bottom I'm going to be putting out tutorials and resources and utilities for it. XD