The Cookie Machine - Click here to drag window

DUMMY TEXT - Real text set in assets/js/theCookieMachine.js

If you can read me, I'm broken!

ToC Skip

Introduction

mserve (Music Server) is firstly a Music Player that entertains with animated graphics, VU meters and scrolling lyrics. Secondly, it encodes CDs with track titles and artwork automatically obtained from MusicBrainz. Finally, mserve automatically downloads lyrics scores from the internet. Lyrics score lines are synchronized by simply clicking each line as it is sung.

Take a quick peek at one of the many merve videos on this website.

Features

mserve has regular features you would expect. This section lists some unique features you might not expect.

Innovative Features Most Applications Don’t Have:

Features Most Music Players Don’t Have:

Multiple Everything

mserve remembers and restores positions and sizes for multiple windows plus a whole lot more.

Under Construction

Under Construction.png
Under Construction.png

mserve is still under construction. Installation requires manually downloading files from GitHub and installing any missing dependencies with apt get install in Debian/Ubuntu or pip install on other Operating Systems.

See the required dependencies for mserve section for more details. For typical power users, many of the dependencies will already be installed.

IMPORTANT NOTES:

mserve Installation

mserve (Music Server) is written in Python. The main program is called mserve.py and can be found in the mserve GitHub Repository ⧉ 🔗.

Copy all files in the GitHub src folder to a new directory on your machine. For example, <HOME>/mserve for Linux, Mac, Chrome OS or Windows Subsystem for Linux (WSL). Unless you are running Ubuntu 16.04 LTS under Extended Security Maintenance (ESM), you will have to make changes to mserve.py and the programs it calls.

For Windows, the installation directory would be <HOME>\mserve As of April 8, 2024 mserve will not run under Windows without modification.

mserve.py is called with m from the command line or a desktop shortcut. It is recommended you start using mserve from the command line to see any error messages that might appear.

m is a wrapper Python script that centers a logo on your screen for a moment while mserve is loaded into memory. Using m instead of mserve.py speeds up loading because mserve.pyc is automatically called and it is half the size. As of August 30, 2023, the former is 765 KB and the latter is 409 KB.

m and mserve.py do not need to be added to your path. You can call them with /path/to/m or /path/to/mserve.py from the command line. If you followed the installation tip above, it would be <HOME>/mserve/m for Linux-like machines, or <HOME>\mserve\m for Windows. As of August 30, 2023, mserve will not run under Windows without modification.

Top ToS Skip

Table of Contents

Top ToS ToC Skip

Music Location Tree

The Music Location Tree is the main window which appears when mserve starts up and it remains until mserve is closed. On startup, all files in the Music Location Tree are rediscovered. New songs since the last time are displayed.

Songs are stored under collapsed Album Names which, in turn, are stored under collapsed Artist Names. The directory format must be:

Only Song Filenames with a music type extensions are included. For example, extensions of .flac, .mp3, .m4a, .oga, .wav, etc, are included.

Here is a sample window with currently playing song highlighted in green.

mserve.py Music Location Tree.png
mserve.py Music Location Tree.png

The first three Artists are “collapsed” which is indicated by the “▶” chevron (A.K.A. “right triangle). When you click the “▶” chevron the Artist is opened and the “▌” chevron is displayed to indicate the Artist is opened. The same chevrons and used for Albums. As songs play and end in mserve, the Album Name and Artist Name are automatically expanded and collapsed to show the green highlight bar.

TIP: Double-click on an Artist or Album to expand and collapse entries underneath.

Songs have a checkbox which are clicked to include or exclude in playlists. The check box is colored solid when “checked” and is hollow when “unchecked”. If a line appears, it indicates the Artist or Album is “tri-state”. This means some songs below are “checked” and some are “unchecked”.

You can check and uncheck individual songs, entire Artists or, entire Albums.

As you check and uncheck individual songs, or entire Artists or, entire Albums a list is built in memory. Then you can Apply or Cancel changes.

New songs are added into the Chronology (Playlist) after the current playing song position.

If you make huge mistakes you can abandon changes with the option “Exit without saving Playlist”.

The Music Location Tree window follows the directory structure of your storage device:

NOTE: “My Music” is an over simplification for the sake of example. You can start m (the splash screen for mserve.py) by typing: m "/mnt/music/Users/Person/Music/iTunes/iTunes Media/Music/"

You can also call m after changing to a music directory. E.G. Type: cd ~/Music/Pink Floyd and press Enter . Then type: m . and press Enter . Note the . specifies the current directory and m must be in your current path. mserve will open and display all the Albums for Pink Floyd.

If you don’t pass a parameter to m it will reload the last location used and continue playing favorites from where it left off.

The top-left corner of the Music Location Tree window contains four dropdown menus; File, Edit, View and Tools. Click on the name and the dropdown menu options appear:

File Dropdown Menu

Some options will be disabled out when they are not applicable. For example, the Save Playlist and Close Playlist options are disabled (greyed out) until a Playlist is opened.

Edit Dropdown Menu

View Dropdown Menu

NOTE: The three SQL views allow the column to be moved. Click and hold the heading to drag the column to a different position.

Tools Dropdown Menu

Sample videos for the Tools Dropdown Menu are available below with detailed explanations.

NOTE: The Enable TV Commercial Buttons and Enable FF/Rewind Buttons dropdown menu options occupy the same space and replace each other when they are clicked. The button selection only effects the the current Playlist. Other locations and playlists maintain their own button selection.

Right-Click Popup Menus

In all mserve windows, you can move the mouse over rows and they are highlighted. You can right click on a row for a context-sensitive popup menu.

When an Artist, Album or Song line in the Music Location Tree window is right-clicked, a context-sensitive popup menu appears. The context-sensitive popup menu changes depending on the line type, which is why it is called “context-sensitive”.

Artist or Album Right-Click Popup Menu

When you click on an Artist or Album, it is expanded and entries beneath are highlighted in yellow. The yellow highlighted entries is a reminder of what will be effected by the next action.

Song Right-Click Popup Menu

When you highlight and right click a song in the Music Location Tree, a context-sensitive popup menu appears.

Information Centre

Just like other applications, mserve uses dialog boxes to display:

However, unlike other applications, mserve also records these in the Information Centre for the duration of your session.

Additionally mserve records system events and user actions where there was no dialog box presented.

The Information Centre is accessed from the Music Location Tree window’s View dropdown menu. It can also be accessed by clicking the light blue thin ruler near the top of the window, as the video below illustrates.

Information Centre Sample Video

Music Location Tree Help Button

mserve windows contain “Help” buttons that:

Help Button Sample Video

Help Button Sample Video Highlights

Top ToS ToC Skip

Automatic Skin Color Based on Artwork

This video shows how the skin changes color:

The album artwork is automatically downloaded from the internet when the CD is encoded. You can also choose artwork from any website and use that instead.

The third pixel to the right and third pixel down set the skin tone when you resize the window that could yield a different skin tone!

Automatic Skin Color Notes

Image For Songs with No Artwork

This .gif (no sound) shows how a custom image is used when a song with no artwork is played:

No Art 5.gif
No Art 5.gif

Show below are the lines you need to change in the mserve.py Python script:

# When no artwork for song use this image file
ARTWORK_SUBSTITUTE = g.PROGRAM_DIR + "Be Creative 2 cropped.jpg"
# "Be Creative 2 cropped.png" is a 4.4 MB image 3120x3120

Copy your image file to the directory where you installed mserve.

NOTE:

The .gif video was created using:

mmm to get the window coordinates. However, you can use wmctrl -lG

byzanz-record -c --verbose --delay=1 --duration=5 --x=3668 --y=481 --width=1506 --height=737 "No Art 4.gif"

Then an on-line .gif optimizer reduced the file size from 33 MB to 22 MB using compression level 35 for “Lossy GIF”.

Top ToS ToC Skip

CD Encoding

Great lengths were taken to ensure currently playing song’s animations never lag even when CD’s are being encoded. This video shows how the music player keeps playing without any lag while a CD is being encoded:

Great lengths were also taken to ensure animations never lag even when focus grabbing dialog boxes require a response. If a focus grabbing dialog box is open when current song ends, the next song is played.

The encoding process uses libdiscid to read the Audio CD’s Disc ID. Then Musicbrainz is accessed to get track listings for Disc ID. It accesses Musicbrainz a second time to get the first recorded release date. It then grabs Album Artwork from the internet.

You can paste album artwork from the clipboard which you previously copied from Amazon, or another website.

How-To Encode a CD Overview

After inserting a CD, click the Rip CD button in the Music Location Tree window. The video below shows how to select encoding format, quality, naming format, album date, artwork, genre, comments and track level overrides to genre:

Encode CD Sample Video Highlights

Album Level Overrides Notes

Notes About Track Names in Bottom Pane

Encoding Metadata Tags

mserve Metadata Tags are displayed with common names. The common names follow the ffmpeg naming conventions:

ffmpeg TAG Description
TITLE Name of the Song
ARTIST Name of band or solo artist
ALBUM_ARTIST Same as ARTIST except for Compilations then “Various Artists”
ALBUM Name of the Album
COPYRIGHT Date the Album (not the song) was released
DISC Disc Number. E.G. single CD is “1/1”. 3 CD set could be “1/3”
TRACK_NUMBER E.G. When 12 tracks, first track “1/12”, last track “12/12”
DATE Song’s first release date in YYYY format. NOT the Album Date!
GENRE E.G. “Rock”, “Soundtrack”, “Country”, etc.
CREATION_TIME Date and time music file created (encoded)
COMPOSER When not specified, defaults to ARTIST
COMMENT One line comment
COMPILATION When value is “1”, folder is /Compilations/
GAPLESS_PLAYBACK “0” = Off, “1” = On. However, mserve doesn’t support it.
ENCODER E.G. “mserve 3.4.3” or “iTunes 11.4.0.18”
DISCID CDDB Free Disc ID
MUSICBRAINZ_DISC MusicBrainz Disc ID

Besides these Metadata tags, mserve SQL stores metadata for:

When mserve first discovers a song it takes a “snapshot” of the file’s:

When file times are updated by the Operating System, they are NOT refreshed in mserve SQL database. A side-effect of this is mserve can reset all files last access time when a program like Rhythm Box reads every song file and resets last access time to current time. See sql.py for examples of “fixing” stuff.

Renaming Artists, Albums and Song files After Encoding

Sometimes the on-line databases contain errors. For example, on a three CD Greatest Hits collection, you will find these Album names:

The last CD of the set doesn’t say [Disc 3] nor does it say Volume 3. It gets confusing when you are viewing the Music Location Tree window.

Use this Function to Rename, Not the File Manager

A file manager will not rename Artists, Albums and Song Titles within the mserve SQL database. If you rename with a file manager, original names will still be in the SQL database. So it is important to use this function instead of a file manager.

Call the function within the Music Location Tree window:

  1. Right click on the third CD Album (Greatest Hits of the 80’s).
  2. Select “Rename Album” from the popup menu.
  3. In the dialog box enter “Greatest Hits Of The 80’s [Disc 3]”
  4. Click the “Apply” button.
  5. A summary dialog box appears, as shown in the next section.

Rename Album Completion

mserve rename Greatest Hits Of The 80's [Disc 3].png
mserve rename Greatest Hits Of The 80's [Disc 3].png

Usage Notes:

Special Notes about ID3 Tags:

Substituting Special Characters in Filenames

When an Artist, Album or Song have these characters in the name:

The character will be replaced with the _ character instead. This is necessary to conform to operating system rules for directory and filenames.

For example, if the names with special characters are:

The file created in Linux will be:

The file created in Windows will be:

Note: In Linux the / character is used to separate directory levels. In Windows the \ character is used to separate directory levels.

Top ToS ToC Skip

Python Modules Dashboard

This dashboard is autogenerated when the website is refreshed. There are more dashboards in The Cookie Machine ⧉ for global Pippim Website maintenance.

Below are all the Python Modules in mserve. Note: Turn your mobile phone sideways (Landscape Mode) to see all columns.

Python Modules used in Pippim mserve Version 3.5.0

Python Module Lines Modified Description
calc.py 260 2024-03-21 16:50:20 Calculator for big numbers
disc_get.py 47 2023-07-16 23:16:08 Get discid of CD
encoding.py 4,065 2023-12-10 11:35:59 Encode (Rip) CD to music files
external.py 617 2024-02-19 08:29:13 Calls to External Programs
global_variables.py 210 2024-03-25 19:32:58 Global variables shared by all modules
image.py 1,800 2024-03-30 20:18:15 Image Processing
location.py 4,876 2024-04-07 12:45:02 Locations of Music Dirs & Devices
m 83 2024-03-11 16:20:23 m - Wrapper for mserve Fast Startup
mbz_get1.py 418 2023-08-21 22:30:50 Get musicbrainzngs ‘release-list’
mbz_get2.py 143 2023-08-22 00:18:06 Get musicbrainzngs artwork
message.py 1,869 2024-03-30 08:25:10 Message Dialog Boxes
monitor.py 887 2024-03-24 09:49:55 Multiple Monitor Management
mserve.py 20,974 2024-04-08 06:17:45 Main mserve Python Module
mserve_config.py 1,323 2024-04-04 17:22:30 Dependencies Checker and Setup
sql.py 4,576 2024-04-07 22:49:03 SQLite3 Interface
timefmt.py 270 2023-10-29 17:40:17 Date & Time formatting
toolkit.py 4,430 2024-04-08 06:19:59 Tkinter Tools and Tooltips()
vu_meter.py 202 2023-09-12 14:12:01 VU Meter processor spawned by mserve.py
vu_pulse_audio.py 802 2023-12-03 08:03:07 Pulse Audio Volume Controls
webscrape.py 1,411 2024-04-06 10:36:59 Webscrape Lyrics from genius.com
x11.py 313 2023-12-31 09:25:08 X11 window client
ALL Modules 49,576    

Table was updated April 8, 2024. The table is autogenerated when refresh.sh ⧉ 🔗 calls mserve_config.py ⧉ 🔗 which writes output to ~/website/programs/mserve_incl.md ⧉ 🔗

There are also some Bash scripts:

Dependencies

Below are the dependencies documented in mserve.py python program. You may already have them installed. The list is for Ubuntu 16.04 LTS using Python version 2.7.12. For Python 3+ versions, substitute python3 below where it says python:

sudo apt install compiz                  # for Hockey (smooth shark move)
sudo apt install dconf-editor            # for Hockey (gsettings)
sudo apt install ffmpeg                  # for artwork, ffprobe and ffplay
sudo apt install gstreamer1.0-tools      # For encoding CDs gst-launch-1.0
sudo apt install kid3                    # Optional for editing metadata
sudo apt install pauvcontrol             # For VU Meters (sound redirect)
sudo apt install pqiv                    # Make transparent Shark (Hockey)
sudo apt install python-appdirs          # Application directory names
sudo apt install python-beautifulsoup    # Scrape Song lyrics
sudo apt install python-gi               # Gnome window functions (newer)
sudo apt install gir1.2-wnck-3.0         # Gnome window functions (older?)
# NOTE: python-wnck not tested but may work instead of gi + gir1.2-wnck-3.0
sudo apt install python-libdiscid        # Get CD's disc ID
sudo apt install python-notify2          # Popup bubble messages
sudo apt install python-numpy            # Installed by default in Ubuntu
sudo apt install python-magic            # Get file type "magic" information
sudo apt install python-musicbrainzngs   # Get metadata for CD
sudo apt install python-mutagen          # Encode and ID3 tags
sudo apt install python-pil              # PIL graphics routines
sudo apt install python-pil.imagetk      # PIL image processing
sudo apt install python-pyaudio          # For background job vu_meter.py
sudo apt install python-requests         # Get Cover Art
sudo apt install python-selenium         # Automated YouTube Playlist play
sudo apt install python-subprocess32     # To compare locations
sudo apt install python-simplejson       # automatically installed Ubuntu
sudo apt install python-tk               # Tkinter (default in Windows & Mac)
sudo apt install wmctrl                  # To move Kid3 or Fishing window
sudo apt install x11-apps                # xwd window dump (screen shot)
sudo apt install xclip                   # Insert clipboard
sudo apt install xdotool                 # To move Kid3 or Fishing window

Additionally, there are external repositories (PPA) that need to be installed.


ffmpeg & ffprobe
============================================================================
The versions released with Ubuntu can be 8 years old. For example, In the
year 2024, Ubuntu 16.04 LTS ESM has ffmpeg version 2.8.17 from 2016. As of
April 2024, stable 6.1 versions of ffmpeg and ffprobe can be found at:

   https://ffmpeg.org/download.html

ffmpeg version 3.1 is minimum version for "loudnorm" filter processing. The
"loudnorm" filter is used to normalize maximum volume levels to 0 dB. You
can install ffmpeg and ffprobe to ~/bin and keep original versions in
/usr/bin.


External Repositories
============================================================================
sudo add-apt-repository ppa:j-4321-i/ttkwidgets  # CheckboxTreeview
# This is necessary for ttkwidgets and ttkcaldenar
sudo apt-get update
sudo apt-get install python-ttkwidgets           # CheckboxTreeview
sudo add-apt-repository ppa:j-4321-i/ppa
sudo apt-get update
sudo apt-get install python-tkcalendar

Finally, there are programs that have no sudo apt install capability such as pulsectl that require git pull command followed by cp command.

As of April 8, 2024, dependencies have to be manually installed. A long term plan is to create an installation script that installs all dependencies automatically. In the short term, development has begun to identify installed versions.

Top ToS ToC Skip

SQL Views

mserve uses proprietary data dictionary technology to quickly view rows in the three SQL tables:

SQL Location Table Data Dictionary Driven Viewer

The SQL Location Table defines the various music locations on your local device, external storage, remote hosts and mobile phones.

mserve uses proprietary data dictionary technology to quickly view rows in the SQL Location Table. Use the View dropdown menu from the Music Location Tree (main) window of mserve. A sample video appears below.

SQL Location Table Viewer Sample Video

SQL Table Viewer Sample Video Highlights

SQL Tables

The popular SQL database engine sqlite3 which is used by your web browser is also used by mserve.

Here are the SQL Tables and Indices that are created in the sqlite3 file ~/
/mserve/library.db:

def open_db(LCS=None):
    """ Open SQL Tables - Music Table and History Table
        Create Tables and Indices that don't exist
    :param LCS: instance of Location() class for lcs.open_code, etc.
    """

    #open_new_db()  # Database 'library_new.db' only used for conversions.

    global con, cursor, hist_cursor, loc_cursor, lcs
    if LCS:
        lcs = LCS  # Locations class

    con = sqlite3.connect(FNAME_LIBRARY)

    # MUSIC TABLE
    con.execute(
        "create table IF NOT EXISTS Music(Id INTEGER PRIMARY KEY, " +
        "OsFileName TEXT, OsAccessTime FLOAT, OsModifyTime FLOAT, " +
        "OsChangeTime FLOAT, OsFileSize INT, " +
        "ffMajor TEXT, ffMinor TEXT, ffCompatible TEXT, " +
        "Title TEXT, Artist TEXT, Album TEXT, Compilation TEXT, " +
        "AlbumArtist TEXT, AlbumDate TEXT, FirstDate TEXT, " +
        "CreationTime TEXT, DiscNumber TEXT, TrackNumber TEXT, " +
        "Rating TEXT, Genre TEXT, Composer TEXT, Comment TEXT, " +
        "Hyperlink TEXT, Duration TEXT, Seconds FLOAT, " +
        "GaplessPlayback TEXT, PlayCount INT, LastPlayTime FLOAT, " +
        "LyricsScore BLOB, LyricsTimeIndex TEXT)")

    con.execute("CREATE UNIQUE INDEX IF NOT EXISTS OsFileNameIndex ON " +
                "Music(OsFileName)")

    # HISTORY TABLE
    con.execute(
        "create table IF NOT EXISTS History(Id INTEGER PRIMARY KEY, " +
        "Time FLOAT, MusicId INTEGER, User TEXT, Type TEXT, " +
        "Action TEXT, SourceMaster TEXT, SourceDetail TEXT, " +
        "Target TEXT, Size INT, Count INT, Seconds FLOAT, " +
        "Comments TEXT, Timestamp FLOAT)")

    con.execute("CREATE INDEX IF NOT EXISTS MusicIdIndex ON " +
                "History(MusicId)")
    con.execute("CREATE UNIQUE INDEX IF NOT EXISTS TimeIndex ON " +
                "History(Timestamp)")
    con.execute("CREATE INDEX IF NOT EXISTS TypeActionIndex ON " +
                "History(Type, Action)")

    # LOCATION TABLE
    con.execute(
        "CREATE TABLE IF NOT EXISTS Location(Id INTEGER PRIMARY KEY, " +
        "Code TEXT, Name TEXT, ModifyTime FLOAT, ImagePath TEXT, " +
        "MountPoint TEXT, TopDir TEXT, HostName TEXT, " +
        "HostWakeupCmd TEXT, HostTestCmd TEXT, HostTestRepeat INT, " +
        "HostMountCmd TEXT, HostTouchCmd TEXT, HostTouchMinutes INT, " +
        "Comments TEXT)")
    con.execute("CREATE UNIQUE INDEX IF NOT EXISTS LocationCodeIndex ON " +
                "Location(Code)")


    ''' For mserve.py rename_file() function to rename "the" to "The" '''
    con.execute("PRAGMA case_sensitive_like = ON;")

    con.row_factory = sqlite3.Row
    cursor = con.cursor()
    hist_cursor = con.cursor()
    loc_cursor = con.cursor()

Pickled Data Files

The pickle data file format allows serialized Python objects such as variables, lists and dictionaries to be stored in non-serialized format on storage devices.

An abbreviation system is used for the filenames below:

For Windows:

For MacOS:

For Linux, ChromeOS, Windows Subsystem for Linux:

Here are the data files (stored in pickle format) created in the ~/.../mserve directory:

One subdirectory is created for every location. E.G. the subdirectory ~/.../mserve/L004 contains:

Within mserve Python scripts, lc.FNAME represents:

Note: When working inside the location.py module, drop the lc. prefix. In the other Python modules, import location as lc is used.

Pickled YouTube Playlists

The directory ~/.../mserve/YouTubePlaylists/ contains:

Generating YouTube Playlists requires some manual steps:

Copy 1

Before copy 1, open playlist in browser and press Ctrl + i . This will open the Console in the Chrome and Firefox web browsers.

let goToBottom = setInterval(() => window.scrollBy(0, 400), 1000)

Copy 1:

Note: You may have to adjust window height and/or console divider to split window between regular browser view and console view.

Copy 2

 clearInterval(goToBottom)
 console.log('\n'.repeat(50))

 let arrayVideos = []
 const links = document.querySelectorAll('a')
 for (const link of links) {
     if (link.id === "video-title") {
         link.href = link.href.split('&list=')[0]
         arrayVideos.push(link.title + ';' + link.href)
     }
 }

 let arrayTime = []
 const spans = document.querySelectorAll('span')
 for (const span of spans) {
     if (span.id === "text" &&
     span.classList.contains('ytd-thumbnail-overlay-time-status-renderer'))
     {
         arrayTime.push(span.innerText.replace(/[^\d:]/g, ''))
     }
 }

 if (arrayVideos.length === arrayTime.length) {
     for (var i=0; i<arrayVideos.length; i++) {
         arrayVideos[i] = arrayVideos[i] + ';' + arrayTime[i]
         console.log(arrayVideos[i])
     }
 }

Copy 2:

Copy 3

 let data = arrayVideos.join('\n')
 let blob = new Blob([data], {type: 'text/csv'})
 let elem = window.document.createElement('a')
 elem.href = window.URL.createObjectURL(blob)
 elem.download = 'my_data.csv'
 document.body.appendChild(elem)
 elem.click()
 document.body.removeChild(elem)

Copy 3:

Move CSV File

Put the following bash / shell commands into a script you can call. The sample file youPlaylistMoveCSV.sh can be copied and renamed.

#!/bin/bash
#  https://www.pippim.com/programs/mserve.html#pickled-youtube-playlists

#    Follow instructions and note "Copy Button" below:

#        STEP 1: Use CTRL+I in web browser
#        STEP 2: Click Button 1 to copy to clipboard "youPlayListScroll()"
#        STEP 3: Go to web browser and use CTRL+V then Enter
#        STEP 3A: Type "allow pasting" (without the quotes) if requested by browser
#        STEP 3B: Wait for web browser to stop scrolling, 1 second per song
#        STEP 4: Click Button 2 to copy to clipboard "youPlaylistCopy()"
#        STEP 5: Go to web browser and use Ctrl+V then Enter
#        STEP 6: Click Button 3 to copy to clipboard "youPlaylistSave()"
#        STEP 7: Go to web browser and use Ctrl+V then Enter
#        STEP 8: Run this bash script youPlaylistMoveCSV.sh
#        STEP 9: Use "View Playlists", select Playlist, View Button

if [ "$#" -ne 1 ]; then
    printf 'ERROR! You must provide the "Playlist Name" in quotes!\n' >&2
    exit 1
fi

if [ ! -f ~/Downloads/my_data.csv ]; then
    printf "ERROR! File ~/Downloads/my_data.csv not found!\n" >&2
    exit 1
fi

cd ~/Downloads
mv -v my_data.csv "$1".csv
cp -v "$1".csv ~/.local/share/mserve/YouTubePlaylists
rm -v ~/.local/share/mserve/YouTubePlaylists/"$1".pickle

Move CSV file:

Processing Playlists Methods

JSON Data Files

Just like pickle data files, the JSON format allows serialized Python objects like lists and dictionaries to be stored in non-serialized format on storage devices.

Unlike the pickle format, the JSON format is human-readable.

You will find JSON data files whenever a location resides on an FTP Host Server like an Android mobile phone.

These are the files you will find in the location subdirectory:

Top ToS ToC Skip

Windows Open Where You Want Them

Most applications always open their windows at the same locations. Then you have to move the windows to where you want them. mserve remembers where you like your windows to be and moves them there.

Besides the dozen or so windows that mserve uses, it also remembers if you use Hockey TV commercial buttons or FF & Rewind buttons instead. It also remembers if you prefer the Chronology (playlist) hidden or shown and the exact second of the last song you were listening to, or even if it was paused.

This is how mserve remembers and restores window positions and sizes:

def save_window_geom(name, geom):
    """
    CURRENT:
        Get geometry for window which was saved on last exit. If no record
        use 100,100 and predefined default width & height. Returns string
        of "width x height + x + y" with no spaces in between variables.
    """

    if sql.hist_check(0, 'window', name):
        sql.hist_cursor.execute("SELECT * FROM History WHERE Id = ?",
                                [sql.HISTORY_ID])
        d = dict(sql.hist_cursor.fetchone())
        if d is None:
            print('monitor.save_window_geom error no History ID:', HISTORY_ID)
            return False
    else:
        # First time add the record
        # sql.hist_add(time.time(), 0, lc.USER, 'window', name, geom,
        sql.hist_add(time.time(), 0, g.USER, 'window', name, geom,
                     'saved on exit, loaded on starting', None, 0, 0, 0.0,
                     "Used in conjunction with 'screen' History Record Id #")
        sql.con.commit()
        return True

    ''' We have the existing history record, simply replace the geometry field '''
    sql_cmd = "UPDATE History SET Time=?, SourceMaster=? WHERE Id = ?"

    sql.cursor.execute(sql_cmd, (time.time(), geom, sql.HISTORY_ID))
    sql.con.commit()

Top ToS ToC Skip

Tooltips Gradually Fade In and Out

A lot of work has gone into crafting the tooltips to delay before gradually fading in. Also, to gradually fade out. And finally, the Tooltip message bubble follows the moving mouse pointer.

Key features of tooltips:

Top ToS ToC Skip

Lyrics Synchronization

After song lyrics have been trained (Time Index assigned to each lyrics line) each line is highlighted as it is sung. Before training, lyrics are auto-scroll based on preset algorithm. Manual scrolling can be turn on to override Auto and Time scrolling.

A sample video is shown below. It shows how the toggle button works between automatic lyrics scrolling and manual scrolling:

Notes:

  1. When video starts with song in orange the default is “Auto Scrolling”
  2. Click button to engage “Manual Scroll”
  3. Now scroll bar appears on right, and you can scroll lyrics
  4. The vido changes to next song in black and the default is “Time Scrolling”
  5. Click the button to engage “Manual Scroll”
  6. Click the button again to reengage “Time Scrolling”
  7. Now each lyrics line is automatically highlighted as it is sung
  8. For Time Scrolling to work you need to train mserve with the timing.

Synchronized Lyrics in Action

This video shows how artwork, automatically obtained from the internet, is animated on your screen:

IMPORTANT: Un-mute video to hear song

This video also shows:

Additional notes:

Top ToS ToC Skip

Basic Time Synchronization

The ☰ (Hamburger) Dropdown Menu is shown below:

mserve lyrics hamburger menu.png
mserve lyrics hamburger menu.png

The same Dropdown menu appears when you right-click on the lyrics score (the song’s lyrics).

A sample video is shown below. It shows how the Basic Time Index feature works. Simply click to highlight and synchronize each lyrics line as it is being sung:

Basic Time Synchronization Sample Video

Basic Time Synchronization Sample Video Highlights

  1. The ☰ (Hamburger) Dropdown Menu where the Basic time index option is picked.
  2. Canceling the Basic time index option once started.
  3. Restarting the Basic time index from the ☰ menu.
  4. Clicking each line as it is sung.
  5. The time indices are automatically saved when the song ends, or you can choose the “Save index” option from the ☰ menu if, you don’t want to wait for the song to end.

Top ToS ToC Skip

Fine-Tune Time Index

Sometimes you just can’t seem to click at the right time using Basic Time Index in the previous section. For those cases the Fine-Tune Time Index window is provided.

Begin Sync option

In the following video notice how the option is included in the ☰ (Hamburger) menu and is selected. The video delays long enough so that you can see all the menu options.

This video shows:

  1. The Lyrics ☰ (Hamburger) Dropdown Menu options
  2. The “Ignore click” option on the menu. This closes the menu which is the same as moving the mouse off the menu and clicking outside the menu.
  3. Access the hamburger menu again and select the “Fine-tune time index” option
  4. The Fine-tune time-index window opens up and pauses the regular music player
  5. Select lyric lines in the lyrics score
  6. Begin sync button. As music plays you can click the line as it is sung
  7. Clicking each line as it is sung is the same behavior as the Basic time index function except that additional details are displayed
  8. Finally, the Fine-tune time index window is closed and regular music player resumes where it was interrupted

80% of lines must be basic synchronized

This screen appears when you have not completed basic time synchronization for at least 80% of the lines:

Basic Time Not Done.png
Basic Time Not Done.png

Fine-Tune Time Index cannot be done until 80% of lines have Basic Time Index completed.

Top ToS ToC Skip

Sample All option

The following video also shows how the Fine-tune time index function is selected. This time the video spends a little time showing you all the buttons in the function.

After turning on sound for the video below and clicking play, make sure you move your mouse outside the video. This way you can see the entire contents underneath your browser’s video control bar.

Video Highlights:

  1. The “Paused” graphic in the regular music player. It is programmatically generated and not an image file that can be changed.
  2. The regular music is resumed and the ☰ menu is used to select the Fine-tune time index option
  3. The Sample all button is selected
  4. The function plays the first second of each line
  5. We noticed at time index 154 seconds the instrumental section was left on too long. This caused the Chorus line and next line to start too late.
  6. Those three lines were selected and Begin sync button is used to fine-tune the timing
  7. When fixing the timing though we clicked too soon rather than
    too late as before. So we click back on a previous line and take a “mulligan”. Then we click again as the music catches up.
  8. Next, the time indices are saved by clicking the Save button
  9. Finally, the function is closed by clicking the Close button and the regular music player resumes play automatically.

Top ToS ToC Skip

Hide/Show Chronology

The Chronology Song List shows:

You can hide the Chronology Song List. This expands the art work and shifts the Lyrics Score beneath the Metadata and VU Meters. The sample video belows shows mserve playing Favorites playlist and the Hide Chronology button being clicked.

Hide Chronology List Sample Video

Hide Chronology List Sample Video Highlights

Note: The software used to create this video, made volume changes jump when volume slider was moved. Volume changes are “as smooth as silk”, when using mserve in real life.

Chronology List Popup Menu

Right-click on any song in the Chronology Song List to:

Top ToS ToC Skip

Locations

When you start mserve, or open and play a different location, music resumes playing where it left off. If music was paused, it is paused at the the same song position when mserve ended.

Locations are the heart of controlling mserve. Locations keep track of where music is stored.

In addition to tracking music on local storage, Locations can access music stored on a File Server or a Mobile Phone.

Sample View Locations Window

Here is a sample View Locations window with the selected location highlighted in green.

mserve View Locations.png
mserve View Locations.png

Mandatory Location Fields. The first two fields are mandatory:

The above sample screen was generated when optional host commands are not installed. 20,000 lines deep inside mserve, are these python tests:

self.nmap_installed = ext.check_command('nmap')
if self.nmap_installed:
   ''' Command 'nc' also required to quickly check if host is up '''
   self.nmap_installed = ext.check_command('nc')
self.ssh_installed = ext.check_command('ssh')
self.sshfs_installed = ext.check_command('sshfs')
if self.sshfs_installed:
   self.sshfs_installed = ext.check_command('fusermount')
self.wakeonlan_installed = ext.check_command('wakeonlan')

Sample Edit Location Window

Here is a sample Edit Location window with the selected location highlighted in green. This location is a “Sleeping Host”.

mserve Edit Location.png
mserve Edit Location.png

Mandatory Location Fields. The first two fields are mandatory:

The remaining fields, starting at Optional Remote Host Name, are optional.

Optional Location Fields:

If the File Server spends most of its life sleeping, mserve can wake it up with a “Magic Packet” over wired Ethernet. Then mserve keeps the host awake by “touching” a specific filename on the server. A special script called mserve_client.sh needs to be running on the host to keep it awake.

Android Wifi FTP Server Host

mserve Wifi FTP Server cropped.png
mserve Wifi FTP Server cropped.png

For wireless location synchronizing with an Android mobile phone, curlftpfs is used instead of sshfs. Older versions of mserve used sshfs but recently Android Wifi SSH Server stopped working on Android 10 mobile phone. On August 30, 2023, mserve was upgraded to use Wifi FTP Server instead.

These notes on Wifi FTP Server were quickly put together to meet August 31, 2023 mserve documentation deadline.

Make sure m or mserve.py is run from the command line when synchronizing phone locations. Extra messages are printed that do not appear in dialog boxes or the Information Centre

Install Wifi FTP Server ⧉ 🔗 from Google Play.

Initially you can accept the defaults for anonymous user and port 2221.

Grant access to the Music folder in Android. Android 10 requires you to press a weird icon to select SD Card instead of Internal Storage before selecting Music folder which exists on both medium.

The Wifi FTP Server screen will stay lit on your phone. Do not switch permanently to another Android App that will dim the screen. If you do, then mserve will report permission errors because host will be down. Of course this only applies when synchronizing. Normal phone usage doesn’t matter about Wifi FTP Server one way or the other.

In mserve Edit Location window go to the Command to wake up sleeping Host field and enter:

curlftpfs -o nonempty,uid=1000,gid=1000,umask=0022,user=android:android phone:2221 /mnt/phone

Above assumes local mount point is /mnt/phone and you have permissions to /phone directory but not /mnt directory which is owned by root.

Above assumes your /etc/hosts contains hostname phone with appropriate IP address such as 192.168.0.11 or whatever static IP address you assigned with the router software.

Above assumes your user id (not user name!) is 1000. Without this, files are mounted as owned by root.

The first time synchronization is run every file is diff checked which takes considerable time. Then modification time stamps are synchronized and subsequent synchronizations are 100 times faster. The first time it takes 1.5 hours to synchronize 4,000 songs over Wifi using curlftpfs. Subsequent synchronizations take 6 seconds for 4,000 songs (when there are no differences).

Known Problems with curlftpfs

curlftpfs chokes on filenames containing # of %.

Quote below from: JackSlateur / curlftpfs ⧉ 🔗

Note:

This is not the official project, which can be found there: http://curlftpfs.sourceforge.net/

I just added some code the correctly handle filename which contains url-special chars (actually, just # and %) by url-encoding them :

% -> %25

# -> %23

Using that, curl will not translate them, and will target the correct filename.

When curlftpfs chokes on a file, mserve transfers the file using ftp to compare the file to the current location.

Occasionally time-out errors are displayed in the console like this:

wait_for_cmd_output() 10 second time-out
Error on file: /mnt/phone/30 Seconds To Mars/A Beautiful Lie/10 A Modern Myth.m4a
Error: Permission denied on 'diff' check return code: 4

wait_for_cmd_output() 10 second time-out
Error on file: /mnt/phone/30 Seconds To Mars/A Beautiful Lie/11 The Battle Of One.m4a
Error: Permission denied on 'diff' check return code: 4

The reason is unknown, but when the next synchronization is run, the error doesn’t appear on the same files. To fix the error, the time-out was increased from 10 to 60 seconds. If more than 60 seconds is needed for the time-out, increase the value on line number 4034 in the file location.py:

if elapsed > 60.0:  # Aug 31/23 WiFi change 10.0 to 60.0 for `diff`

Optional Remote Host Support

There are two types of remote hosts supported:

Linux File Server

To debug keeping host awake, run the following commands on the host and client:

HOST - Open a terminal and enter command which runs forever:

mserve_client.sh -d

CLIENT - Open a terminal, and paste below, replacing <HOST> with Host name:

while : ; do ssh <HOST> "cat /tmp/mserve_client.log" ; sleep 60 ; done

When the commands fusermount, nmap, nc, ssh and sshfs are installed, extra fields appear for location details. See screenshots below.

Ensure Music files are mounted on the Host. If you’re using Windows iTunes, and need to mount in Linux, you may need something like:

sudo mount -t auto /dev/sdb1 /mnt/music

See: https://help.ubuntu.com/community/MountingWindowsPartitions

Make sure your SSH is using the standard port 22. Otherwise, you will have to open location.py and search on 22 > and 22/tcp.

Never use -odebug option for sshfs from mserve because it will lock up. Use debug only from command line for testing. mserve will not let you define this debug option.

More details: https://help.ubuntu.com/community/SSHFS

Android Mobile Phone

Older versions of mserve supported ssh on mobile but the Android application Wifi SSH Server no longer supports sshfs. In September 2023, mserve switched to FTP for mobile phones, with local filename caching.

Install Wifi FTP Server ⧉ 🔗 from Google Play.

More details on using FTP are above.

Initially FTP startup time was 5 minutes in mserve. Subsequently file-caching was utilized and startup time is now a few seconds for 4,000 music files.

Problems still exist using curlftpfs which hasn’t been updated in 9 years. For example, go to Location Music Tree (main window) and select the View Dropdown Menu. Then select the SQL Music Table option, and then the Update Metadata button. All files with # in the name are reported as: “1) Not a music file.”. These files:

Test Remote Host Status Display

These steps are followed when a remote host is tested:

Note: When synchronizing a remote host location, the same test is automatically run.

Sample Synchronize Location Test Host Video

The above video demonstrates:

Synchronize Location

When locations are synchronized, new files are NEVER added, and old files are NEVER deleted. If you would like to test the function first, review the script test_for_synch.sh.

Sample Synchronize Location Window

Below is a sample screen where the selected location is highlighted in green:

mserve Synchronize Location.png
mserve Synchronize Location.png

The steps below describe what happens when the “✔ Synchronize” button is clicked.

Synchronize Location Actions

A quick test is made to see if files have the same modification time and same size. If so the next file is checked.

If files have the same size but different times, the diff command is used to test every byte to see if they are different. If the files are identical, the modification times are set the same to the oldest modification time. The rationale is a new location was created, files were copied from an original location and the operating system reset the modification time to the current time.

If the files have the same time but different sizes or different contents, an error message is displayed because mserve has no clue which direction to copy files in.

If files are different, then the file with the newest modification time is copied over the oldest.

Before copying you are always given a chance to review action plans.

Below is an example of the action plan window:

mserve Synchronize Actions.png
mserve Synchronize Actions.png

The “Action” column states the reason for updating is that the file size has increased. This was due to album artwork being added to the source files.

Here’s a list of actions. All appear in “Action” column except those denoted as “hidden”:

The “Src”/”Source” location is the location currently opened in mserve. The “Trg”/”Target” location is the location that was picked to synchronize with.

The “OOPS” should never appear but is technically possible if another job is running that updates files in the Source or Target Location.

Note: For Remote Hosts, the linux cp command is used not the SSH scp command.

For Android mobile phones the last modification time (used to compare files) may not work properly. In this case, mserve creates a virtual modification time file to track what modification times should be.

Analyze Volume

ffmpeg is used to analyze volume levels. Two functions are provided:

Under Development.

Sample Analyze Volume Windows

Under Development.

Analyze Volume Actions

Under Development.

Top ToS ToC Skip

Playlists

Playlists are stored by location. Each location can have an unlimited number of playlists. Each playlist can have an unlimited number of songs, but only songs from the current music location.

Below is a sample Playlist Maintenance window from the Rename Playlist function:

Sample Rename Playlist Window

mserve Rename Playlist.png
mserve Rename Playlist.png

The Playlist name cannot be blank and must be unique per location. A warning is issued when the Playlist name has been used in another location.

The Playlist description is optional.

The Device location code is automatically assigned.

The columns for “Song Count”, “Size of Files” and “Duration” are automatically calculated as songs are selected and deselected in Music Location Tree checkboxes.

Five functions share the same Playlist Maintenance window:

YouTube Video with LRC Synchronized Lyrics

The screen below shows LRC (Synchronized LyRiCs) playing in mserve with the YouTube window dragged overtop.

mserve LRC Synchronized Lyrics
mserve LRC Synchronized Lyrics

NOTES:

mserve Smart Play YouTube Playlist

The screen below shows the mserve Smart Play Playlist window with a YouTube Playlist window next to it.

mserve YouTube Playlist compared to YouTube
mserve YouTube Playlist compared to YouTube

Top ToS ToC Skip

Hockey TV Commercial Buttons

When mserve starts, the Rewind 10 seconds and Fast Forward 10 seconds buttons are active. You can change these buttons to Stanley Cup Hockey Playoff TV Commercial buttons.

Basic Time Not Done.png
mserve hockey buttons.png

.gif (there is no sound) Highlights:

This .gif also shows how the Show/Hide Chronology button places song lyrics in a suitable location when the artwork size changes.

Using Hockey TV Commercial Button

This video shows what you see and hear when you click one of the Hockey TV Commercial Buttons.

Video Highlights:

Hockey TV Commercial Button Without compiz

This video shows what you see and hear when you click one of the Hockey TV Commercial Buttons and compiz code is commented out.

Hockey TV Commercial Sample Video Highlights

Although it doesn’t look as nice with the shark “jumping” between monitors, it works more reliably. To disable gsettings and the compiz option to disable place windows (which allows for smooth shark movement across monitors but leads to instability), open the file image.py and around line 1119 comment out the code:

# Removing "place" from gsettings allows smooth shark movement over
# monitors. However there are screen resets with disappearing windows
# for a couple seconds from time to time. Keeping "place" has shark
# stop at monitor border then "jump" into the next monitor.
'''
if "'place', " in self.old_compiz_plugins:
    self.place_in_plugins = True
    override = self.old_compiz_plugins.replace("'place', ", '')
    #print('override:', override)
    self.set_gsettings(override)
'''

mserve Volume During TV Commercials

This image shows mserve volume (ffplay) is at 60%:

hockey_volume.png
hockey_volume.png

When TV commercial ends, TV volume returns to 100% in 10 steps over 1/2 second. Music is paused in mserve and volume is reduced from 60% to 25% in 10 steps over 1/2 second.

As of June 11, 2023, mserve TV Commercial volume is initialized at 60% on line 401 in mserve.py:

TV_BREAK1 = 90          # Hockey TV commercial is 90 seconds
TV_BREAK2 = 1080        # Hockey TV intermission break is 18 minutes
TV_VOLUME = 60          # Hockey music play 66% of mserve volume level
TV_SOUND = "Firefox"    # Hockey broadcast is aired on Firefox browser

60% was found to be a suitable volume level for CBC broadcasts of the NHL Stanley Cup Playoffs. YMMV. Note that TV_VOLUME is a short name. A full name would be something like: “mserve volume when playing during hockey game muted TV commercials”.

FYI the “ALSA plug-in [python2.7]” sound processor is used by mserve to display the VU meters (vu_meter.py). Configuring the VU Meters using system output loopback to input stream is described in the next section.

Configure mserve Volume During Muted TV Commercials

To set the mserve volume during muted TV commercials, click the Edit dropdown menu and select Volume During TV Commercials:

mserve volume for tv commercials.png
mserve volume for tv commercials.png

This window changes the program variables shown in the last section.

Every location and every playlist within every location has a separate volume control for mserve when TV commercials are muted by mserve.

Top ToS ToC Skip

VU Meters

The VU Meters show in mserve need to be configured using pavucontrol (Pulse Audio Volume Control). Before we dive into pavucontrol though, the screenshot below shows how Ubuntu 16.04 displays sound output devices. This is for comparison purposes and to show you a screen you may already be familiar with.

Ubuntu 16.04 Sound Settings Panel.png
Ubuntu 16.04 Sound Settings Panel.png

In the example mserve application used on this webpage, the first output device called “GM204 High Definition Audio Controller” is usually used for sound output. This belongs to a 50” big screen Sony TV with good sound system including a sub-woofer. The soundcard (chipset) is located on a Nvidia 970M discrete graphics card.

The middle output device is “HDMI / DisplayPort - Built-in Audio”. This sound device is 43” 4K TCL / Google (Android) TV. The sound card is built into Intel Skylake (Thunderbolt 3) USB-C to HDMI converter plug.

The bottom output device is “Speakers - Built-in Audio”. These are sub-par speakers on a laptop. The only time they would be used if the laptop was unplugged from the local network (LAN). The soundcard is built into onboard Intel chipset.

pavucontrol Sound Output Loopback to Microphone

In order for VU Meters to work in mserve, The Pulse Audio Volume Control GUI application (pavucontrol) is used.

Pulse Audio Volume Control Sound Output

pavucontrol output devices.png
pavucontrol output devices.png

This screenshot shows the Pulse Audio Volume Control’s Output Devices Tab. In this example, the output was changed from the first device to the Built-in Audio speakers. The changes were made from the Ubuntu 16.04 Sound System Settings panel show in the previous section.

Notice the thick progress bar. It displays the sound playing on the output device and jumps very quickly. Progress bar activity is how you can confirm the active output device is selected.

When the output device is changed, the recording device must be changed for the VU meters to display the correct output sound device. (See the next section)

Pulse Audio Volume Control Recording Tab

pavucontrol recording tab.png
pavucontrol recording tab.png

This screenshot shows the Pulse Audio Volume Control’s Recording Tab.

Notice the thick progress bar at the bottom indicates no sound is being recorded. That is because it is listening to the wrong stream: “Monitor of GM204 High Definition Audio Controller Digital Stereo (HDMI)”. This stream is for the 50” Sony TV connected to Nvidia 970M card.

Remember in the last screenshot we used Ubuntu’s sound setting to change output from 50” TV to built-in laptop speakers.

Pulse Audio Volume Control Change Recording Source

The sample video shows the Pulse Audio Volume Control Recording Tab.

The stream “Built-in Audio Analog Stereo” is selected. Notice the thick progress shows sound volume level changes.

Then the stream name “Monitor of GM204 High Definition Audio Controller Digital Stereo (HDMI)” is selected.

When you change the output device loopback to recording; YOU MUST RESTART mserve. Otherwise the VU meters will merely be blank.

Credits and References:

Top ToS ToC Skip

Tools Dropdown Menu Examples

The Tools Dropdown Menu is found on the Music Location Tree Window.

These are the options in the Tools Dropdown Menu:

Sample Make LRC For Checked Songs Video

The Make LRC For Checked Songs feature will create an LRC (.lrc - synchronized lyrics) file for ever checked song in the Music Location Tree. This only applies to songs that have a lyrics score web scraped and where you have clicked on 80% of the lines to synchronize them.

Make LRC For Checked Songs Sample Video Highlights

NOTES:

Big Number Calculator Sample Video

Dealing with large numbers can make the mind numb. For example, the average drive of 1 Terabyte is 1000000000000 bytes. An average video of 4 Gigabytes is 4000000000. An average song of 8 Megabytes is 8000000 which is a little easier to type and read. However, it’s much easier with the Big Number Calculator.

In this video, we have 3,800 songs and we need to buy a USB stick to hold the music. What size should we buy?

Big Number Calculator Sample Video Highlights

NOTES:

Top ToS ToC