m3u uri handling + radiobrowser + store webstream info in db

[ik666]

PR #4572 follow up

Implements RadioBrowser Metadata lookup.
Added MediaTableContainerFiles to track files included in a container.
Store WebStream media info into the database.
Use of FFmpegParser to parse media info.
Implements WebStreamParser to parse headers of an http streams.
Added possibility to add playlist in playlist.

[SurfaceS]

One question: is -> transcode to PCM = transcode to LPCM ? I think this using LPCM as a default stream encoding for audio files is not such a good idea, because in real world audio devices PCM is more common and should therefore be the most compatible format.

Maybe we should rename isTranscodeToPCM to isTranscodeToLPCM and isTranscodeToWAV to isTranscodeToPCM

@ik666 I was not yet their when this was implemented.
I just tell you what it do in FFmpegAudio.

		if (renderer.isTranscodeToMP3()) {
			cmdList.add("-f");
			cmdList.add("mp3");
			cmdList.add("-ab");
			cmdList.add("320000");
		} else if (renderer.isTranscodeToWAV()) {
			cmdList.add("-f");
			cmdList.add("wav");
		} else { // default: LPCM
			cmdList.add("-f");
			cmdList.add("s16be");
		}

According to http://trac.ffmpeg.org/wiki/audio%20types

The default for muxing into WAV files is pcm_s16le

s16le           PCM signed 16-bit little-endian
s16be           PCM signed 16-bit big-endian

so, I presume :
TranscodeToWAV is PCM signed 16-bit little-endian onto a wav container.
default (not mp3 or wav) is PCM signed 16-bit big-endian in raw format.

[SurfaceS]

Unsure if it's done: If UMS delivers "audio/wav" as mime type to the renderer, FFMPEG must encode a WAV header at the beginnen of the stream, so the renderer can extract the stream type.

In my point of view, audio/wav should have wav container.
raw format PCM signed 16-bit big-endian should be audio/L16.

Edit : it is the case :

				if (isTranscodeToWAV()) {
					matchedMimeType = HTTPResource.AUDIO_WAV_TYPEMIME;
				} else if (isTranscodeToMP3()) {
					matchedMimeType = HTTPResource.AUDIO_MP3_TYPEMIME;
				} else {
					// Default audio transcoding mime type
					matchedMimeType = HTTPResource.AUDIO_LPCM_TYPEMIME;

audio/wav, audio/mpeg, audio/L16

[SurfaceS]

@ik666
I added a PR on your git :
ik666#91

Now streams are parsed then we should be able to know if transcode is needed or not.

• a lot work on storing metadata the right way

[ik666]

@SurfaceS I think this is working so far, but needs a tweak. From what I see, the transcoding section is "optimized" for video transcoding only. I think we have 2 use cases :

1. container having audio and video streams
   If you have a video container with audio streams inside, the audio streams get transcoded to LPCM since most AV receiver are able to handle raw data audio (i.e. LPCM encoded 7.1 streams) for home theater systems. This is currently the config-default and just fine I believe.

2. container having only audio streams
   In this case we usually have a 2 channel audio stream only. The renderer usually accept only PCM, not LPCM. So you have to disable the transcode to LPCM checkbox in the audio transcoding section for audio streams/files to work correctly.

You can check this even in the web ui. If you navigate to this entry in the web-ui

#EXTM3U

#RADIOBROWSERUUID:960e4940-0601-11e8-ae97-52543be04c81
#EXTINF:1,Ibiza Global Radio
http://ibizaglobalradio.streaming-pro.com:8024/

having the transcode to LPCM option checked => no sound in the web-ui
having the transcode to LPCM option disabled => radio station is playing

But I believe if LPCM is disabled the video files (having DTS, ATMOS, etc.) won't play any audio any more since the AV receiver usually requires LPCM audio streams. The user would need to switch between the two option, in regard of the file type ...

Maybe we should have a "video files transcoding" configuration section and a "audio files" configuration section to separate the config between those two container types.

Quick fix would be default transcoding to PCM if we have no video streams in the container (maybe ignoring if LPCM is checked) or moving this option to a renderer configuration section.

[SurfaceS]

container having audio and video streams -> FFMpegVideo engine
container having only audio streams -> FFMpegAudio engine

having the transcode to LPCM option disabled

you should set this on the renderer configuration.
TranscodeAudio = WAV or TranscodeAudio = MP3

Maybe we should have a "video files transcoding" configuration section and a "audio files" configuration section to separate the config between those two container types.

Quick fix would be default transcoding to PCM if we have no video streams in the container (maybe ignoring if LPCM is checked) or moving this option to a renderer configuration section.

I think this PR is huge and have a lot of changes.
Maybe it should be merged first, then we can open another PR for next thinking.

[ik666]

you should set this on the renderer configuration.
TranscodeAudio = WAV or TranscodeAudio = MP3

Maybe you can help: Which renderer configuration uses the web-gui (player)?

[ik666]

@SurfaceS Probably another issue, not directly related to this PR. For the above example "Ibiza Global Radio" delivers a MP3 stream. The renderer has a supports line for that

Supported = f:mp3                       m:audio/mpeg

However, a transcoding engine is set. Looks like the class StoreItem line 484 sets a compatible engine

		// Try to match an engine based on mediaInfo information and format.
		resolvedEngine = EngineFactory.getEngine(this);

without matching against the supported line of the renderer? Probably at the start of the public Engine resolveEngine() method this check ought to be done.

Edit: Actually FFMPEG does a good job in reading the radio station info's. Transcoding to PCM is actually not such a bad idea.

[ik666]

container having audio and video streams -> FFMpegVideo engine container having only audio streams -> FFMpegAudio engine

I believe the problem is, that both engines (audio & video) use the global transcode to LPCM configuration option. It's not configurable for "only audio" or "only video".

[ik666]

I think this PR is huge and have a lot of changes.
Maybe it should be merged first, then we can open another PR for next thinking.

Yeah. I tried a few thing, but let's first merge this and see, how to proceed ...

[SubJunk]

I added the file suggested at #4572 (comment) and I get an error:

DEBUG 2024-06-14 18:50:34.544 [MediaScanner File Parser] Analyzing file my_internet_radio_station2.m3u
TRACE 2024-06-14 18:50:34.544 [MediaScanner File Parser] Matched format PLAYLIST to ".m3u"
DEBUG 2024-06-14 18:50:34.545 [MediaScanner File Parser] Reading m3u playlist: my_internet_radio_station2.m3u
DEBUG 2024-06-14 18:50:34.545 [MediaScanner File Parser] Adding playlist entry: [https://dancewave.online/dance.mp3,Dance Wave!]
TRACE 2024-06-14 18:50:34.545 [MediaScanner File Parser] Matched format WEB to "https://dancewave.online/dance.mp3"
TRACE 2024-06-14 18:50:34.545 [MediaScanner File Parser] Adding new child "Dance Wave!" with class "WebAudioStream"
TRACE 2024-06-14 18:50:34.545 [MediaScanner File Parser] Adding "Dance Wave!" to transcode folder for engine: "null"
ERROR 2024-06-14 18:50:34.545 [MediaScanner File Parser] Adding container entry 24127 for 24130 
TRACE 2024-06-14 18:50:34.545 [MediaScanner File Parser] Store do not yet contains MediaInfo for https://dancewave.online/dance.mp3
TRACE 2024-06-14 18:50:34.546 [MediaScanner File Parser] Adding audio from the database: Audio Id: 0, Codec: -, Bitrate: 128, Channels: 2, Sample Frequency: 48000 Hz
TRACE 2024-06-14 18:50:34.546 [MediaScanner File Parser] Searching AUDIO_METADATA with "HikariProxyPreparedStatement@2041336265 wrapping prep200495: SELECT * FROM AUDIO_METADATA WHERE FILEID = ? LIMIT 1 {1: CAST(24127 AS BIGINT)}"
TRACE 2024-06-14 18:50:34.546 [MediaScanner File Parser] Searching VIDEO_METADATA with "HikariProxyPreparedStatement@2106837540 wrapping prep200496: SELECT * FROM VIDEO_METADATA WHERE VIDEO_METADATA.FILEID = ? AND (VIDEO_METADATA.IMDBID IS NOT NULL OR VIDEO_METADATA.TMDBID IS NOT NULL) LIMIT 1 {1: CAST(24127 AS BIGINT)}"
TRACE 2024-06-14 18:50:35.350 [NetworkConfiguration Watcher] Checking network configuration changes
ERROR 2024-06-14 18:50:37.799 [MediaScanner File Parser] Database error while trying to add parsed information for "https://dancewave.online/dance.mp3" to the cache: Cannot invoke "net.pms.media.audio.MediaAudio.setBitRate(int)" because the return value of "net.pms.media.MediaInfo.getDefaultAudioTrack()" is null
TRACE 2024-06-14 18:50:37.799 [MediaScanner File Parser] Queuing background WebStream lookup for https://dancewave.online/dance.mp3
INFO  2024-06-14 18:50:37.799 [MediaScanner File Parser] New file my_internet_radio_station2.m3u was detected and added to the media store
INFO  2024-06-14 18:50:37.799 [Lookup RadioBrowser Metadata background worker 2] paging=Paging(offset=0, limit=1)
DEBUG 2024-06-14 18:50:37.799 [Lookup RadioBrowser Metadata background worker 2] Connecting to https://nl1.api.radio-browser.info/json/stations/byuuid/962cc6df-0601-11e8-ae97-52543be04c81
DEBUG 2024-06-14 18:50:37.799 [Lookup RadioBrowser Metadata background worker 2] POST WWW-Form-UrlEncoded body: offset=0&limit=1

[SurfaceS]

@SubJunk
I fixed that on the @ik666 PR ik666#92
Just have to wait he merge it :)

[ik666]

@SubJunk Now it's merged

[SubJunk]

Thanks, I will test it. Can you resolve the conflicts?

[ik666]

@SubJunk Conflict is resolved ... but now we have a compile error ... @SurfaceS maybe you can have a look, since it look like a small change you made.

[SurfaceS]

@SubJunk fixed