Creating simple Choir Rehearsal files with Lilypond

Ever since I have started to prepare and lead choir projects, I have been trying to supply the singers with choir rehearsal files – recordings or computer generated audio files with all voices and one/their part/voice highlighted – for their individual preparation. With Corona and the need for online preparation and self-study, this concept has grown to become even more important.

One can find dozens of such rehearsal files for famous pieces as videos on Youtube. Here is an example (created by somebody else), the tenor part of Locus Iste (Anton Bruckner):

However, sometimes you cannot find existing rehearsal files, esp. if you’re planning to perform lesser known songs and pieces. In these cases, you might want to create such rehearsal files yourself.

Being very fond of (the free software) Lilypond for typesetting scores – for my choirs as well as for other projects – I have come up with a simple workflow to create audio and video rehearsal files. Since I use Lilypond to typeset most of the pieces anyway, I can re-use my existing source files. Hence, creating audio and videos tutorials becomes a matter of minutes.

Here is how I do it. As a result I will have one video file per voice with

  • an audio stream containing all voices as string sounds and the rehearsing voice as trumpet sound (louder)
  • images of the score with the rehearsing voice highlighted in red

Let’s start with a simple example, the beginning of Palestrina’s Adoramus Te. The PDF originally typeset with Lilypond looks like this:

The Lilypond source looks quite simple. It contains the preamble, a few global definitions, and the music with one voice (sopranos, altos, tenors, basses) per staff. Using a layout as well as a midi definition, I can create a pdf (score) and midi (audio) file in one step. Here is the complete Lilypond source for the part shown above:

\version "2.18.2"

\paper {
  top-margin = 1\cm
  indent = 0\cm
  system-system-spacing = #'((padding . 0) (basic-distance . 18))
}

\header {
  title    = "Adoramus Te"
  composer = "Palestrina"
  tagline  = ""
}

global = {
  \key c \major
  \numericTimeSignature
  \time 4/2
}

words = \lyricmode {
  Ad -- o -- ra -- mus te Chri -- ste,
  et be -- ne -- di -- ci mus ti -- _ bi.
}

sopranMusic = \relative c'' {
  gis\breve
  gis1 a1~
  a1 a1
  gis\breve
  a2.( b4 c1)
  b1 r2 b2
  b2 b2 c1~
  c2 b2 b1
  a\breve
  g\breve
  g1 r1
  \bar "|."
}

altMusic = \relative c' {
  e\breve
  e1 e1~
  e1 d1
  e2.( d8 c b1)
  c2( d2 e2 fis2)
  g1 r2 g2
  g2 g2 g1~
  g2 g2 g1
  f1( e2 f4 e4)
  d\breve
  e1 r1
  \bar "|."
}

tenorMusic = \relative c' {
  b\breve
  b1 c1~
  c1 a1
  b1 e1~
  e2( d2 c1)
  d1 r2 d2
  d2 d2 e1~
  e2 d2 d1
  d2( c4 b4 c2) f,2
  g\breve
  r\breve
  \bar "|."
}

bassMusic = \relative c {
  e\breve
  e1 a1~
  a1 f1
  e\breve
  a\breve
  g1 r2 g2
  g2 g2 c,1~
  c2 g'2 g1
  d4( e4 f4 g4 a4 b4 c2~
  c2 b4 a4 b1)
  c1 c,1
  \bar "|."
}

\score {
  \new ChoirStaff <<
    \new Staff = "sopranos" <<
      \new Voice = "sopranos" {
        << \global \sopranMusic >>
      }
      \new Lyrics = "sopranos"
      \context Lyrics = "wSopranos"   \lyricsto "sopranos" \words
	>>
    \new Staff = "altos" <<
      \new Voice = "altos" {
        << \global \altMusic >>
      }
      \new Lyrics = "altos"
      \context Lyrics = "wAltos"   \lyricsto "altos" \words
    >>
    \new Staff = "tenors" <<
      \new Voice = "tenors" {
        \clef bass
        << \global \tenorMusic >>
      }
      \new Lyrics = "tenors"
      \context Lyrics = "wTenors"   \lyricsto "tenors" \words
	>>
    \new Staff = "basses" <<
      \new Voice = "basses" {
        \voiceTwo
        \clef bass
        << \global \bassMusic >>
      }
      \new Lyrics = "basses"
      \context Lyrics = "wBasses"   \lyricsto "basses" \words
    >>
  >>
  \midi {
    \tempo 2 = 92
  }
  \layout { }
}

Creating the audio files

Let’s start with creating the audio files. Remember, the audio will contains all voices as strings – to get an impression of the harmonics – as well as the rehearsing voice as trumpet which is a little bit louder. This will give the rehearing singers a clear impression of their part.

In the following example, let’s assume we want to create a rehearsal file for the tenors.

Lilypond let’s you define with sounds/instruments to use when creating a MIDI file. The code below shows you, how to do this:

\new ChoirStaff <<
  \new Staff = "sopranos" \with {
     midiInstrument = "string ensemble 1"
     midiMinimumVolume = #0.5
     midiMaximumVolume = #0.7
  } <<
     \new Voice = "sopranos" {
     << \global \sopranMusic >>
  }
  \new Lyrics = "sopranos"
  \context Lyrics = "wSopranos"   \lyricsto "sopranos" \words
 >>
 …

As you can see, we will use the “string ensemble 1” sound for all voices. It is a stringy sound which can be used very well to create an overall impression. Also, we will decrease the volume a bit.

Please note, that I use these settings for all voices!

Now, let’s add an additional staff and voice for the loud trumpet part. Since, we have used special indepedent declarations for all voices, we can simple re-use the tenorMusic commands so that we don’t need to repeat anything. Just add another staff:

\new Staff = "trumpet" \with {
  midiInstrument = "trumpet"
  midiMinimumVolume = #0.8
  midiMaximumVolume = #0.9
} <<
  \new Voice = "trumpet" {
  << \global \tenorMusic >>
}
>>

With this approach, I can create audio files for the other voices simply by replacing \tenorMusic with \altMusic etc.

Creating the score with highlighted voices

For the visual part of the rehearsal files, I want the rehearsing voice printed in red. Also, I want the pdf to roughly have the size of a display with only one system per page. Hence, I will use a special paper size/format and a series of color definitions.

On the main level, I use this command:

#(set-default-paper-size "c6" 'landscape)

For the colored output, I use these definitions within the music:

tenorMusic = \relative c' {
  \override NoteHead.color = #red
  \override Stem.color = #red
  \override Accidental.color = #red
  \override Dots.color = #red
  \override Beam.color = #red
  \override Slur.color = #red
  \override Rest.color = #red
  b\breve
  b1 c1~
  c1 a1
  …

The output will look like this.

Creating the video files

As a final step, I want to put the score and the audio output together. For this, I will use OBS Studio. It is another open source software, which can be used to record your desktop session. Simply open the pdf files in full screen (presentation) mode, play the midi file and manually flip the pages whenever your audio stream reaches a new part of the piece.

Unfortunately, this is the most time-consuming part. However, it is a good opportunity to learn all voices 🙂

One can simple create a tile frame (as pdf or image) and use it to start the video in order to give your rehearsal file a professional touch. You can also start with a few explaining words, if you want to pass hints to your singers.

Have fun with Lilypond and this simple way of creating rehearsal files for your choirs and singers. Please let me know, if you have questions or remarks.

Control Smart Home Features with (Satechi®) Bluetooth Button

Sometimes you need another button to control one specific feature in your smart home where no button/control element had been planed. Naturally, digging holes through existing walls in order to lay a new cable is not an option in most cases. I faced this scenario when trying to add another doorbell to my front door. I quickly came up with the idea to use a Bluetooth device to send a signal to some kind of controller – a Raspberry PI in my case – and trigger my KNX-based bell from there.

satechi_bt_home After doing some browsing, I decided to use a Satechi® Bluetooth Button, type Home. It looks quite nice, without any fancy design, and comes with a 3M sticker, so you can put it nearly everywhere. Unfortunately, my existing R-PI (controlling most parts of my KNX installation) was out of range, hence I had to set up a new one for the sole purpose of reacting to the Bluetooth device. The new R-PI then calls a small REST service on my old R-PI, which then triggers the door bell via a signal on the KNX bus.

bluetooth-r-pi-knxHere is a quick list of steps that I took in order to get my new door bell running:

  • Connect R-PI with Bluetooth to my home WIFI network
  • Connect/Pair Bluetooth Button to R-PI using bluetoothctl command (how to)
  • Run a small script to watch for Bluetooth connections using bluetoothctl
  • Upon connection: Call REST service in KNX R-PI

I can confirm the Satechi® Buttons to work flawlessly with the Raspbian OS installed on my PI. Of course, there is a little delay between pushing the button and the bell to ring due to the latency of Bluetooth and WIFI. However, it is quite okay for this scenario. I am pretty sure I will use the same set up for further controls, such as a central off (when leaving the house).

Controlling Home Automation via Telegram Bot

We have been moving to a different house with a KNX-based smarthome installation lately. I have spent quite some time coming up with my own visualisation and automation, mainly using a Python implementation relying on the KNXD (a fork of the well-known EIBD) software for Linux.

When I read about the new Telegram API for Bots, I got excited about the idea of creating a bot for my home automation installation. Using simple commands, I wanted to switch on/off lights, control the temperature, get status messages …

telegram_knxd

Using the Telegram documentation and my PI-based main KNX controller, it was quite easy to come a with a first prototype, which I can use to switch on and off devices. The screenshot is in German, but you can basically see how I use it to switch on my lights and some outlets. More advanced commands will follow soon. Since I have a smart watch (LG Urbane) which I can use to send Telegram messages via voice input, I can even use it as a voice-to-KNX interface without developing a special (watch) app, just relying on existing technology. (Unfortunately, I am cannot start a new chat with my bot for now, but this is a problem of the watch’s OS.)

I am quite happy with this first version and can’t wait to enhance it. Also, I am considering using this “human to machine interface technology” in enterprise/business apps. What do you think?

 

Install KNXD on Raspbian (8)

I am currently in the process of planning the KNX set up for our new home. I have decided to use a Raspberry PI 2 (Model B) with Raspbian OS for the more fancy stuff, such as visualization, logging of measured values, automatic jobs etc. Communication with the KNX bus will be handled by KNXD.

The KNXD git hub page offers a small shell script that can be used — in theory — to install the software on Debian based system (Raspbian is based on Debian). Unfortunately, a few packages are missing in the requirements section (apt-get install).

I have added these packages creating a new version of the KNXD install script specifically for Raspbian OS. My PI runs on Raspbian 8, however, it should work with other releases as well.

#!/bin/bash

# first, install build tools and get the source code
sudo apt-get install git-core build-essential debhelper cdbs autoconf automake libtool libusb-1.0-0-dev libsystemd-daemon-dev dh-systemd
git clone https://github.com/knxd/knxd.git

# knxd requires libpthsem which unfortunately isn't part of Debian
wget https://www.auto.tuwien.ac.at/~mkoegler/pth/pthsem_2.0.8.tar.gz
tar xzf pthsem_2.0.8.tar.gz
cd pthsem-2.0.8
dpkg-buildpackage -b -uc
cd ..
sudo dpkg -i libpthsem*.deb

# now build+install knxd itself
cd knxd
dpkg-buildpackage -b -uc
cd ..
sudo dpkg -i knxd_*.deb knxd-tools_*.deb

Creating PDF documents in OpenUI5 apps

I have been a fan of OpenUI5 (or UI5 in general) ever since my first encounters with it in late 2013. Thus, I have been trying to do some of my work using OpenUI5, and, hence, I have decided to write a few blog posts about UI5 and about how certain things can be achieved. I have mainly been focusing on new apps for our company and myself, since most of our customers have not yet switched to using mobile enabled applications heavily or rather rely on the existing apps from SAP itself. I usually have a PHP-based back-end using JSON as data model provider, since a full SAP stack is an overkill in most cases.

When implementing business apps, creating PDF reports/documents that can be printed, sent via email or stored in an electronic archive often is a required feature. This can be achieved with two different approaches.

  1. It is possible to create the PDF using JavaScript libraries right on the (mobile) device. A common library for generating PDFs in JavaScript is jsPDF. This approach doesn’t require extra communication with the back-end or any back-end at all. On the downside, jsPDF doesn’t seem to be very robust when working with images. Thus, this approach is mainly fitting for small apps with simple output.
  2. The second option is producing the PDF document on the back-end. For PHP-driven services, fpdf is a very good tool for that. Just like jsPDF, fpdf is free software. In my tests, it proved much more reliable when producing complex output using images etc. For this approach, a new script/service has to be called. I found it to be a convenient solution to pass the required data (from the UI5 data model) as a JSON string (JSON.stringify) to the PHP service.

Below, you will find an example for generating PDFs directly on the mobile or desktop device using JavaScript and jsPDF and for creating the documents in the back-end with PDF and fpdf.

jsPDF: Document generation in the (mobile) app

Let us create a small app with input fields and a button for generating a PDF document. The PDF doc will contain the input value to show you, how you can connect your app data to the output process.

Example app

The UI5 coding for our apps looks like this:

<!DOCTYPE html>
<html>
  <head>
	<meta http-equiv='X-UA-Compatible' content='IE=edge' />
	<meta http-equiv='Content-Type' content='text/html;charset=UTF-8'/>
	
	<script id='sap-ui-bootstrap' type='text/javascript' src='https://openui5.hana.ondemand.com/resources/sap-ui-core.js' data-sap-ui-theme='sap_bluecrystal' data-sap-ui-libs='sap.m,sap.ui.commons'></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/jspdf/1.1.135/jspdf.min.js"></script>
	<script>      
		var oModel = new sap.ui.model.json.JSONModel({ name: "Name", age: 30 });
		var p = new sap.m.Page( {
          title: "Generate PDF",
          content: [
            new sap.m.Label({ text: "Name:" }),
            new sap.m.Input({ value: "{/name}" } ),
            new sap.m.Label({ text: "Age:" }),
            new sap.m.Input({ value: "{/age}" } ),
            new sap.m.Button({ text: "Generate PDF", press: function(evt) {
              var n = evt.getSource().getModel().getProperty("/name");
              var a = evt.getSource().getModel().getProperty("/age");
              
              var doc = new jsPDF();
              doc.text(20, 20, "This is " + n);
              doc.text(20, 30, "She/he is " + a + " years old.");
              doc.save('sheet.pdf');
            }})
          ]
        } );
      
      var app = new sap.m.App({ pages: [ p ] }).placeAt("content");
      app.setModel(oModel);
	</script>
	
	</head>
	<body class='sapUiBody'>

<div id='content'></div>

	</body>
</html>

As you can see, pushing the button “Generate PDF” will trigger the document output directly in the app. Therefore, a new object of type jsPDF has to be instantiated providing methods for adding content to the PDF document. jsPDF comes with a lot of examples showing you how to create bigger and more complex documents.

You can give the app a try on jsBin.

fpdf: Document generation in the back-end

Let us take the same app from above, but this time the PDF document will be generated in a PHP-based service.

<!DOCTYPE html>
<html>
  <head>
	<meta http-equiv='X-UA-Compatible' content='IE=edge' />
	<meta http-equiv='Content-Type' content='text/html;charset=UTF-8'/>
	
	<script id='sap-ui-bootstrap' type='text/javascript' src='https://openui5.hana.ondemand.com/resources/sap-ui-core.js' data-sap-ui-theme='sap_bluecrystal' data-sap-ui-libs='sap.m,sap.ui.commons'></script>
    
	<script>      
		var oModel = new sap.ui.model.json.JSONModel({ name: "Name", age: 30 });
		var p = new sap.m.Page( {
          title: "Generate PDF",
          content: [
            new sap.m.Label({ text: "Name:" }),
            new sap.m.Input({ value: "{/name}" } ),
            new sap.m.Label({ text: "Age:" }),
            new sap.m.Input({ value: "{/age}" } ),
            new sap.m.Button({ text: "Generate PDF", press: function(evt) {
              var n = evt.getSource().getModel().getProperty("/name");
              var a = evt.getSource().getModel().getProperty("/age");
              
              window.open("http://some.where/?name=" + n + "&age=" + a, "_blank");
            }})
          ]
        } );
      
      var app = new sap.m.App({ pages: [ p ] }).placeAt("content");
      app.setModel(oModel);
	</script>
	
	</head>
	<body class='sapUiBody'>

<div id='content'></div>

	</body>
</html>

Pushing the button “Generate PDF” this time opens a new window/tab with the specified location passing the form data as HTTP GET parameters. Usually, you will pass bigger amounts of data to the output processing and might consider using HTTP POST. So, what does the back-end script look like:

<?php require('fpdf.php'); $name = $_GET['name']; $age = $_GET['age']; $pdf = new FPDF(); $pdf->AddPage();
$pdf->SetFont('Arial', 'B', 12);
$pdf->Cell(40, 10, 'This is ' . $name . ' aged ' . $age);
$pdf->Output();
?>

The PHP script simply fetches the two parameters (data), creates a new object of type FPDF and uses a few – easy to understand – methods to put some content into the document. Again, fpdf comes with a variety of examples and tutorials to show you how to use it. It takes some time to get used to the positioning of elements etc. But once you have got acquainted to fpdf, it’s quite easy to create large document with different types of content.

In this post, I have shown two different approaches on where to create PDF in (mobile) apps that are based on UI5. Both example were kept as simple as possible ignoring some best practices for UI5. Of course, there are different libraries for PDF output and the same approaches apply for word documents etc. If you know better tools for the job – other than jsPDF and fpdf – feel free to leave a comment. Also, if you’re trying to implement PDF as part of (mobile) apps and require help, please do not hesitate to get in touch.

WebUI components for ITSM

I am currently supporting a SAP Solution Manager ITSM project which requires a lot of modification to the standard ITSM WebUI components. Thus, I have created tiny overview of the most important BSP components in this area. I would like to share this overview here, in case anybody else is looking for one of the specified WebUI elements:

Header comp. Search comp.
Ticket (general) AIC_INCIDENT_H AIC_INCIDENT_S
 RfC AIC_CMCR_H AIC_CMCR_S
 Change AIC_CMCD_H AIC_CMCD_S
 Task BT125H_TASK
 Incident AIC_INCIDENT_H AIC_INCIDENT_S
 Service Request AIC_INCIDENT_H AIC_SRVREQ_S

If I missed an important component, please let me know.

Convert saplogon.ini to SAP Java PlatinGUI connections

I like to use my native Mac OSX system and the Java PlatinGUI for working and developing with SAP whenever I can. Of course, I know that the native Windows GUI is way more advanced (because better taken care of by SAP), but I hate to boot my Windows VM every time I want to do something SAP-related. Since I work for multiple clients having multiple SAP installations (ERP, CRM, Solution Manager, …), I want to have my SAP connections in synch. For my Windows systems – yes, I do have a few of them – I share the saplogon.ini via Dropbox. However, that doesn’t help me having the same connection entries in my Mac Java SAPGUI (PlatinGUI).

Hence, I wrote a tiny (and not very fancy) Python script to convert the saplogon.ini entries to the connection strings used in the Java SAPGUI’s connections file (which on Mac OSX can be found in the user’s Application Library folder). It doesn’t do much and it is not very robust against invalid input, but I would like to share it here, so anybody can use it for their own purposes. If you have an advice on how to improve the script radically or on how to support more input information, please let me know.

#!/usr/bin/env python

import sys, os

try:
    fname = sys.argv[1]
    fh = open(fname, 'r')
except:
    print sys.argv[0], ' &lt;path/saplogon.ini&gt;'
    sys.exit(-1)

data = fh.readlines()
fh.close()

# initialize entry data base (100 records)
entries = []
for i in range(100):
    entry = {'description': '', 'server': '', 'router': '', 'database': ''}
    entries.append(entry)

for line in data:
    data = line.strip()
    if data != '':    # ignore empty lines
        if data.startswith('[') and data.endswith(']'):
            key = data.lower()[1 : len(data) - 1]
        else:
            parts = data.split('=')
            item = parts[0]
            value = parts[1]
            try:
                nr = int(item[4 : ])
                entries[nr][key] = value
            except ValueError:
                pass

# now print connection strings for Java GUI
for e in entries:
    if e['description'] != '':
        comp = '/' if e['router'] != '' else ''
        connStr = e['description'] + ':conn=' + \
            e['router'] + comp + \
            '/H/' + e['server'] + \
            '/S/32' + e['database'] + '&amp;expert=true&amp;wan=true'
        connStr.replace(&quot;/H//H/&quot;, &quot;/H/&quot;)
        print connStr

Citrix Receiver not working with Mac OS X Yosemite

With the update to Mac OS X Yosemite, my Citrix Receiver–which I need for accessing customer systems–suddenly stopped working. I do not know if or why the problem is caused by the update, but it stopped working on both–my iMac and my Macbook Air–at the same point of, meaning with the Mac OSX update. For those Mac and Citrix users having the same problem, I want to provide a quick solution.

The reason for Citrix Receiver not working properly are invalid file permissions in the app specific folder in $HOME/Library. This can easily be fixed, changing the permissions with this simple Terminal command (quick and dirty setting the specific directory to writeable for everyone):

chmod 777 $HOME/Library/Application\ Support/Citrix\ Receiver/Modules

SAP CRM – How to debug BOL objects

With SAP CRM and BOL (Business Object Layer) / GenIL (Generic Interaction Layer), SAP introduced an object-oriented modeling and programming framework. It takes some time to get used to this approach – and knowing OO techniques is most certainly very helpful there – but once you know how to work with BOL/GenIL some development tasks become very easy and straightforward.

With the GenIL BOL and – even more – with the GenIL model browser, SAP delivers tools to analyze and work with BOL objects without requiring custom development. Using these tools, you can browse your BOL model and determine the attributes and relations per object.

However, sometimes you do want to check you objects’ structure directly at run-time, thus in the ABAP debugger. This short post will show you, where to find the appropriate information using the data browser in the debugger. In detail, I will explain how to

  • Access the attributes of a BOL object
  • Find all related objects
  • Find all entities belonging to a BOL collection

 

To start browsing your BOL object, double-click on a BOL entity or collection. You will see something like this:

BOL entity

BOL entity

 

What you see here is the structure (classes, attributes) of a BOL entity of type BTAdminH. To navigate to the (plain) attributes, you must open (double-click) CONTAINER_PROXY -> DATA_REF, then doube-click the object type in the view below.

Data ref object type

Data ref object type

 

You will see your actually BOL type now (BTAdminH in our example) and be able to navigate one step down via ATTRIBUTE_REF.

BOL object

BOL object

 

Once again, you will have to double-click on the object type, then you will see the BOL attributes as you would for any other ABAP run-time or dictionary structure:

BOL attributes

BOL attributes

 

To find about the object related to the current BOL object, go back one step and click on the RELATIONS attribute.

Relations

Relations

 

Double-clicking this entry will display a table with all relations (relation types) as well as a list of related entries per type.

Related objects

Related objects

 

You can use the OBJECTS field to navigate to the related objects.

In most cases, you will start with a collection of BOL objects. Moving from a collection to the objects is possible via the ENTITY_LIST attribute. It is a table with all entities belonging to the current collection.

BOL collection

BOL collection

 

From there, you can use the navigational paths described above. Have fun exploring your BOL objects and collections using the ABAP debugger.

Dynamic Include Texts (IDs …) in SAP Smartforms

I have decided to open a new section in my blog: Time and again, I stumble over SAP related problems and issues that consume quiet a large amount of my (precious) time doing debugging and bug fixing. Sometimes the SCN is a helpful source of information, sometimes it is not. For the issues that I had to fix without finding helpful resources in SCN or on the internet in general, I have created the SAP category in my blog, where I will write short posts on the original problem and the solutions that I have discovered.

The problem that I spent half of my morning on, had to do with SAP Smartforms. It is – generally – possible to create a text object containing the contents of a standard text element. In its most simple form, the text element is given through a static tuple: text name, text object, text ID, and language.

Smartform Text Element Static

It is also possible, to determine the text name dynamically in ABAP coding, e. g. in the Initialization routine, and use a variable as element property. If you want to do this, you have to change the property to dynamic (see red box in the picture below) and put the variable name between &.

Smartform Text Element Dynamic

Now, if you have correctly defined and set your variable, it should work as expected. It should!

For me, it didn’t work. Instead the text name was always empty. After some debugging, I found that the value got lost during a move operation/routine that is executed before the form is really generated.

perform %move using %textkey-name ‘&GV_TEXTID&’.

Further research showed that this routine checks the dictionary object and the variable length. If the value used in the smart form coding is longer than the dictionary element type for the text name field, the value is – not truncated but – set to SPACE. Don’t ask me why the SAP developer chose to go that way; it was surely giving me some headaches 😦

Hence, the solution would be to use global variables that have exactly the same type and field length as the text element property you want to set dynamically!