It is the doers, the creators, the makers, the engineers, and yes, even the hustlers that motivate us, that inspire us to develop Superpowered tools that aren’t simply more, but paramount, actually extend the makers’ creative and productive capabilities – allowing them to create and make things real – profoundly shaping them, the builders, to build things that weren’t possible without Superpowered technology.

–The Superpowered Love Letter to Developers, Hackers, Doers, Makers and Creators

Hi Patrick,
Just wanted to thank you and Gabor for taking the time to develop (and share) Superpowered. I am not even sure how I came across your SDK, but I honestly didn’t believe it was real. I got to check it out over the last few days, and you have restored my hope for mobile music experiences. There are like a million things I could say about all of my frustrations, compromises, etc. over the last three years…but all I really want to say is thanks (seriously, thank you, this is huge).
john

As far as my frustrations, I got into Android and audio development at the same time (~3 years ago). I worked my way through all of the Android audio classes (SoundPool, MediaPlayer, AudioTrack), on to OpenSL for a little while, and then eventually settled for AudioTrack while using MediaCodec to transcode any compressed audio files.

While this solved my issue of being able to play compressed audio formats while still having access to the PCM audio data, I was never able to avoid large latencies, under runs, etc. With this setup, I could hack together a few audio effects (beat rolls, poorly designed filters, gates, rewinds, panning) but it still wasn’t what I imagined

From everything I read, it seemed like using the NDK was the way to go, but I had previously found OpenSL to be difficult to work with (unlike Superpowered, which I got up and running on my lunch break), and from what I could gather, other than lower latency, OpenSL didn’t make it any easier for me to develop audio FX.

And for the last half of a year, that is where I had plateaued…I considered taking a deeper dive into audio development to make my own library, but kept putting it off because I wasn’t sure that the likely huge amount of time and effort would actually give me any better results…which is unfortunate, because I had been so inspired by the music experiences I thought were possible with a pocket-sized computer that had motion and location sensors plus the ability to access all sorts of external data in real-time. But thanks to your work, I am happily back at it again!

From ‘it still wasn’t what I imagined’ to ‘I am happily back at it again!’.

When we hit our marks, when we deliver on our promises, when we super-power our creators, our hearts swell, not only in pride, but in gratitude too.

Thank you for super-powering Superpowered.

The post App Developers’ Testimonials about Superpowered Audio Engine appeared first on Superpowered Low Latency Audio for Android and iOS.

[Click here to jump to technical details on Superpowered HTTP live streaming for Android and iOS]

Android HTTP Live Streaming?

Though Android is the most widely distributed mobile OS, Android HTTP Live Streaming (“HLS”) is not of the same quality as HLS on iOS.

Though newer versions of the Android OS tend to perform somewhat better, according to JWPlayer, Android fragmentation is still a problem and depending on the Android version, HLS may:

*not be supported at all (Android 2.3, Gingerbread)
*cause devices to crash (Android 3.0, Honeycomb)
*lack seeking ability, feature distorted images, and restart videos completely when in fullscreen (Android 4.0, Ice Cream Sandwich)
*lack seeking ability, and throw an error and crash when in fullscreen (Android 4.1, Jelly Bean)

Though excessive buffering resulted in the crashing noted above, the tablet-only OS Honeycomb 3.0 was an attempt at HLS-capability.

The introduction of Android 4.4 KitKat included an update to the platform’s HLS support to a superset of HLS specification version 7 (version 4 of the protocol), yet as the Android Open Source Issue Tracker makes clear, the Android-HLS issue is still far from solved despite the improvements made to KitKat.

Knowing this doesn’t make it any easier on developers looking to utilize HLS streaming in their mobile apps and proves frustrating, considering the wide array of device/Android version combinations in the Android ecosystem.

Developers, then, are left with two alternatives: attempt to build an implementation for your app from scratch, or utilize a third-party SDK that solves the Android HLS problem for you.

In many ways, the struggle for seamless HLS streaming on Android is similar to the struggle for low latency Android audio—but that’s not the only thing the two problems have in common.

They share a common solution as well: Superpowered Audio technology.

But before we detail Superpowered HLS Streaming for Android and iOS, we will describe a typical HLS streaming implementation for Android.

Generally, implementations are made up by using multiple, different external libraries for each layer:

Network Communication: HTTP download, Transport Layer Security for HTTPS.
MPEG Transport: parsing and demuxing.
Audio Decoding: MP3 and AAC.
Media Segment Files: M3U8 parsing.
Audio Processing: resampling.

Inevitably, this makes for severe interoperability and efficiency problems as each library is optimized locally, and kludged together, hence, the streaming performance cannot be optimized holistically.

In other words, you cannot take a Ford engine, put it on a Tesla chassis with a Porsche suspension and steering mechanism from a Trabant and expect any sort of reliable performance. You need to integrate and unify the entire stack.

Introducing

Superpowered HTTP Live Streaming HLS for Android and iOS

The Superpowered HLS Streaming Audio for Android and iOS unites the entire stack of:

Network Communication: BSD Network Socket communication, HTTP Client and Parser, TLS implementation for HTTPS (TLS parser, AES encryption [ECB, CBC, CFB128, CFB8, CTR], X509 parsing [and ASN1], RSA cryptography (PKCS1v1.5), arbitrary-precision arithmetic, CTR DRBG, MD5, SHA1, SHA224, SHA256, SHA384, SHA512 [and HMACs])
AES-128 decryption
MPEG Transport Stream Parser
MP3 and AAC-LC decoding
Caching, efficient memory handling and adjustable download management
M3U8 parsing
HLS decoder to manage all playback integration into the Superpowered Advanced Audio Player with time stretching, pitch shifting, seeking.

This way, with a Superpowered binary, the entire HLS stack runs inside the mobile app – with zero 3rd party dependency libraries, completely independent of Android or iOS versions and fragmentation – is how one achieves stable and reliable HLS Audio Streaming on Android.

The Superpowered HLS player supports:

VOD, live or event streams
AAC-LC or MP3, in audio files or MPEG-TS files
byte ranges
AES-128 encryption
bandwidth measurement, automatic stream switching (can be turned off)

The Superpowered HLS player can be set to three unique download strategies:

Download all remaining content after the playhead.
Download everything: if all download finished after the playhead, will download everything in front as well.
Seconds ahead of playhead strategy: set it to 20 for example, and it will not download more than 20 seconds ahead of the playhead to save bandwidth for the mobile user.

The Superpowered SDK contains a HTTP Live Streaming example project for iOS and Android that shows you how to create a simple HLS audio player with seeking, buffering indicator and time display. Also, choose from multiple live and vod streams, multiple download strategies and enable time stretching.

Please note: Per the license governing Superpowered technology; Superpowered HLS technology must be licensed and is not available for free, unlike the Superpowered SDK.

HLS Streaming Background

Implemented in 2009 by Apple as part of their Quicktime, Safari, OS X, and iOS software, HTTP Live Streaming (HLS) has become the standard media streaming communications protocol for adaptive bitrate (ABR) video on mobile, tablets, and desktop.

HLS supports both live broadcasts and video on demand (“VOD”). Because HLS detects bandwidth capacity, adjusting video quality based on bandwidth available at the time, HLS quality tends to higher—i.e., less buffering and interrupted playback—than streaming a static file at a single bitrate.

Why choose HLS over other streaming technologies?

Because unlike other streaming technologies (eg Adobe Media Server), it makes use of well-known Hyper Text Transport Protocol (“HTTP”). HLS works by breaking a stream into several small HTTP downloads. Rather than loading the entire file at once, each download loads a short chunk of a potentially unbounded transport stream. As the stream is played, the client is able to dynamically switch between several alternate streams—each containing the same data, but encoded at different bitrates—so that the streaming session adapts to the available bandwidth.

The post HTTP Live Streaming (HLS) Android Audio & HLS Android Example appeared first on Superpowered Low Latency Audio for Android and iOS.

This is a tremendous update for Superpowered. We have something for everyone with the latest package:

NEW FEATURES

Introducing Superpowered Dynamics with 3 new features: a compressor, a limiter and a clipper.

The new Superpowered Clipper class implements a hard-knee clipper with adjustable threshold and maximum decibel settings. You can prevent clipping in your audio stream with this super lightweight feature, which runs 3 times faster than a simple filter!

The new Superpowered Compressor is a classic all-round compressor with very low distortion, a nice curve and many settings: input and output gain, dry/wet, attack and release times, ratio, threshold. It also has an adjustable high-pass filter for the sidechain! The Superpowered Compressor needs less than 1 kb of memory and has 0 latency (of course!). It’s very CPU friendly, requiring just 5x CPU compared to a SuperpoweredFilter.

The new Superpowered Limiter provides amazing sound even when you are crushing it to -40 db. You can set the ceiling, threshold and release. It’s very friendly with any device with very low memory usage (below 1 kb), and it uses just 2.5x CPU compared to SuperpoweredFilter (great power savings!).

Support for the Native Instruments Stems format with super high performance. To re-create the exact same sound designed in the Stems Creator tool by an audio engineer or musician, the sound of the SuperpoweredCompressor and SuperpoweredLimiter is closely matched to the sound of the original compressor and limiter by Native Instruments.

We have no doubt, that the SuperpoweredDecoder, SuperpoweredCompressor and SuperpoweredLimiter supercharges the Native Instruments Stems format with much higher processing speed and lower battery usage.

The SuperpoweredAnalyzer can find and return with the stream of bass and mid keys, to extract bass and melody notes from the source audio stream.

The new SuperpoweredWaveform class offers an even higher performance way over the SuperpoweredAnalyzer to return simpler waveform data, based on the peak volume.

IMPROVEMENTS

Xcode 7 support with separately built iOS and OSX binaries.

The SuperpoweredFlanger uses the SuperpoweredClipper now to prevent audio spikes.

The SuperpoweredDecoder returns with various Native Instruments Stems data (colors, names, settings). You can set which stems track should it decode. These changes are reflected in the SuperpoweredAdvancedAudioPlayer class as well.

The offline example project has been extended with a simple decode >>> filter >>> save-to-WAV example.

ANDROID IMPROVEMENTS

64-bit Intel Android support.

Android Studio does not offer complete native/C++ support with static libraries, still. (We checked, that’s the official statement by it’s creators too.)

We tried the new “experimental” Gradle system and hacked really hard to find a way. Having the nice new C++ support and even building with Superpowered is possible, so it’s almost there. But building for multiple architectures with static libraries is not possible with the experimental Gradle system currently.

SuperpoweredAndroidAudioIO is now lock free to prevent dropouts due priority inversion on poorly configured Android devices. It also has an alternative “separate processing thread” option for very old or very badly configured Android devices.

Updated Android example projects to 32-bit ARM, 64-bit ARM, 32-bit Intel and 64-bit Intel architectures, with a little path setting help for Windows users.

The post Audio Compressor, Limiter and Clipper for Android and iOS: Superpowered Audio Engine Update appeared first on Superpowered Low Latency Audio for Android and iOS.

Xamarin is a popular cross-platform mobile app development platform that helps developers create iOS, Android, Mac and Windows apps in C#.

Because Xamarin is written C# and Superpowered is written in C++, a Xamarin developer would need to create a binding to the Superpowered Audio libraries.

One of the developers in the Superpowered Dev community, Bryon Baker, has done just that, and moreover, has open-sourced his implementation.

Bryon writes:

The Xamarin Spectrum Analyser is a small project that demonstrates how to bind C# managed code with the SuperpoweredSDK. It is not an attempt to provide a binding of the complete SDK, but rather demonstrates all of what you need to bind any of the SuperpoweredSDK class library to Xamarin.

The single notable missing binding is the binding to SuperpoweredIOSAudioOutput. This is a complex binding that I did not need it for my initial implementation; I plan to build this later (or will happily accept contributions :-).

The X-Code and Xamarin projects give you everything you need to extend the binding and enjoy the awesomeness of SuperpoweredSDK with cross-platform code!

The source code can be found here: https://bryonbaker@bitbucket.org/bryonbaker/xamarin-spectrum-analyser.git

If you have any questions please feel free to contact me.

Detailed information follows:

X-Code

Calling the C and C++ class library

In order to bind to Superpowered you need to wrap up the C++ code in a way that enables it to be called from managed code. The information on how to do this is actually hard to find but basically there are two things working against you here:

1. C++ uses mangled names which means you need to create a map file to find the code you want to call.
2. You cannot create C++ objects from C#.

To work around this you reimplement each C++ method as a simple C function. The impact of this is that you need to pass object references into functions that need to call instance methods.

For example, the following two functions are part of the reimplementation of the SuperpoweredBandpassFilterbank class:

Constructor: extern “C” SuperpoweredBandpassFilterbank* SuperpoweredBandpassFilterbank_Create(int numBands, float frequencies[], float widths[], unsigned int samplerate);
Instance method: extern “C” void SuperpoweredBandpassFilterbank_SetSamplerate( SuperpoweredBandpassFilterbank *pObject, unsigned int sampleRate );

There are two things going on here:
1. extern “C” ensures that the function uses standard cdecl and does not mangle the function name.
2. The C++ object wrapper needs to pass the object reference in and out in order to construct the object and then call its methods.

WARNING: IT IS CRITICAL THAT YOU DO NOT ALLOW AN EXCEPTION TO PASS FROM UNMANAGED CODE TO MANAGED CODE. It is for this reason that the catch(…) is used and instead a diagnostic message is written to stderr. I think a good enhancement would be to add a result code to the C functions that can be tested in C# and then a call to get the error details.

Calling Objective-C
As I said, I have not bound SuperpoweredIOSAudioOutput yet – but have demonstrated what is required by binding to the SuperpoweredFrequencies demo. From an X-Code perspectiveyou do not need to do anything except make sure it is built into the library – the heavy lifting for this is done in C#. The Xamarin utility Objective Sharpie is used t parse the Objective-C headers and create an ApiDefinition.cs and StructsAndEnums.cs class files. If you take the time to learn the notation I suggest writing them yourself.

Building the X-Code Project
To build the X-Code project you need to use the command line and run “make.” This will create the libSuperpoweredXCodeWrapperSDK.a library that is needed for the Xaramin solution.

Xamarin C#
In the Xamarin Solution there are a couple of parts to the binding. To call the C or C++ code in SuperpoweredSDK you need to use pInvoke. To call Objective-C you need to use a C# binding. The solution therefore contains three projects.

Project 1: SuperpoweredSDKXamarinWrapper – demonstrates how to wrap the C and C++ Superpowered library.
Project 2: SuperpoweredSDKBinding – demonstrates how to bind to the SuperpoweredFrequencies demo in Objective-C (this is where the binding to SuperpoweredIOSAudioOutput will go).
Project 3: XamarinSpectrumAnalyser – the project that demonstrates the implementation of the binding.

The code here is pretty self explanatory and the only thing worth noting is this is where we convert our “flattened” C++ code back into classes. Here I have basically reimplemented the original C++ interface in C# so you can use SuperpoweredBandpassFilterbank as per the current documentation.

Thanks to Bryon’s heavy-lifting, the source code can be found here: https://bryonbaker@bitbucket.org/bryonbaker/xamarin-spectrum-analyser.git.

The post Working with Audio in Xamarin with Superpowered Audio Engine appeared first on Superpowered Low Latency Audio for Android and iOS.