Testing SpectraLayers 11 Pro for dubbing on older Anime (Macross Robotech) This software is awesome!

SpectraLayers
1

Hello!
I needed a mini project to get to learn SpectraLayers Pro 11 and I chose dubbing Macross into Latin American spanish LAT. This dub does not exists. Mine is sourced from the Robotech LAT dub from a TV transmission and the original Macross in japanese on BR.
While I have a few comments about SpectraLayers Pro 11 the results are impressive! Check them out here (USA may be blocked from viewing it):

I’ll post all my tips and tricks when I remember them, but the main one is that Unmix Chorus module works great at separating main vocal track from secondary voice/music tracks.
Also, Transfer function is amazing but only allows about 8 different tracks in the pulldown menu which was not well thought out. So keep your tracks tidy.
Regarding the stability of SpectraLayers Pro 11 I have to say I’m more than impressed. In 5 days it never crashed on me. :stuck_out_tongue:
Aside from two phrases sourced from other parts of the anime few edits were made to adjust the dialogue. No extra FX or postprocessing was made.

I’ll post more tips and tricks and comments a bit later.

Saludos a los chilenos del foro.

2

Hello!
Tips regarding SpectraLayers Pro 11 for extracting audio tracks from Anime. Consider Macross had a sountrack with vocals, instruments and SFX that usually ducked under the main voice track.

  1. One module is generally not enough for this kind of material. Each step is not the same but first I used “Unmix Song” to get the instruments out. With the instruments and some SFX out of the way the “Unmix Chorus” does a good job of separating main voice track with the music vocals but it will get confused with ducking. You will have to clean up the start and end of the voice track.
  2. Sometimes the modules will give “confused” output and you can play around with “Unmix Crowd Noise” and “Unmix Noisy Speech”. You may have to merge back and process again.
  3. Results of a process can change slightly depending on selected region.
  4. SpectraLayers Pro 11 is extremely stable. I pushed it and it never crashed. but always save your projects. Also, there is no versions of projects yet nor backups.
  5. The project file seems self contained, with everything in it, so use fast M.2 storage.
  6. Use caution when doing too many processed as the selected track may have shifted during landing. You can end up with a Groups selected and you “Can’t unmix a group”.

Odd behaviour:

  1. The Target of the Transfer function is set every time you change the track or group and will change on you. Remember to always reset it or you will end up with audio where you did not expect it.
  2. At this time Target layers for the Transfer Function is maxed out at 10, and because tracks can have the same names it can get confusing. The pulldown menu will only show you the first 10 track names with no scroll bar, no grouping nor track color indicator.
  3. Ctrl Z command does not work on everywhere you expect, for example, changing track names.
  4. I’d like it very much if scrolling left and right was Cubase-like. Moving with the mouse feels a bit clumsy. Before I knew it it took me to the end of the project and couldn’t get back quickly. I think we are missing a follow playback head function like Cubase.
  5. I tried exporting consolidated layers with the Export menu. I was expecting each file to be exactly the same length but it is not like that. This can be confusing when verifying many tracks.
  6. Need more colors for tracks. White over lime green offers no contrast.
  7. Unit Grid visualization is forgotten on exit.
  8. The region under the filenames feels a bit underused and in high resolution screens the markers are barely visible. Spectral Region is not even there, and colors would help.
  9. I wish there was a “Cut and Paste” transfer tool for large selected regions, or maybe I missed it.
  10. Harmonic Selection with high overtone values kills performance but is still stable.
  11. High brush values work but give stutters so some portions are missed and stay unaffected when moving too quickly.
  12. I find no use for the 3D view or maybe I’m just using it wrong, but it does look fancy!
  13. When selecting regions I was not able to move a frequency up, only down.
  14. Some modules seem to process audio forward and not backward. For instance, it detected voice and will start working there but there may have been voice before and won’t recheck what was already processed.

I’d be curious to discuss the internals with the developers since I’m interested in AI as well as less traditional uses of technology. I also know the processed results can sometimes be unexpected. For instance, some high frequency white noise explosions ended up in the Bass track when using “Unmix Song”, and some others ended on the Guitar track where I think the module thought they were distorted guitars.

I’m a sound designer and PC user, I’d love it if SpectraLayers had MetaSynth-like capabilities. SpectraLayers is somewhat close on some spectral aspects. A piano roll for using the transform tool for us designers? how about rotate selected region? So many cool possibilities!

Thank you for your time!

3

[quote=“irregular, post:2, topic:924133”]5.
I tried exporting consolidated layers with the Export menu. I was expecting each file to be exactly the same length but it is not like that. This can be confusing when verifying many tracks.
[/quote]

Yeah! I noticed this but it never bothered me because the start of the layers was never affected (meaning the overall phase relationship remained the same).

[quote=“irregular, post:2, topic:924133”]6.
Need more colors for tracks. White over lime green offers no contrast.
[/quote]

If that is the case then I(if I were the developer) I would implement a full color palatte generator.

[quote=“irregular, post:2, topic:924133”]7.
Unit Grid visualization is forgotten on exit.
[/quote]

So is selections or any other selections you make. Selections are only saved within a different tier system within the project, meaning if you open a brand new project and make any selection and save that project and reopen that project then that selection is lost. Most applications don’t work that way and that interferes with the INTUITIVE workflow of how most people use applications.

[quote=“irregular, post:2, topic:924133”]9.
I wish there was a “Cut and Paste” transfer tool for large selected regions, or maybe I missed it.
[/quote]

To be honest, I believe the new “transfer tool” was only implemented as a “new feature” within Spectralayers 11 in order to entice customers into buying Spectralayers 11. The “transfer tool” should’ve been implemented within the subset tool selection menu (where you have add to selection substract from selection intersect to selection)

[quote=“irregular, post:2, topic:924133”]10.
Harmonic Selection with high overtone values kills performance but is still stable.
[/quote]

I agree that a lot of Spectralayers needs heavy optimization overall.

[quote=“irregular, post:2, topic:924133”]12.
I find no use for the 3D view or maybe I’m just using it wrong, but it does look fancy!
[/quote]

I disagree. It looks nice!

[quote=“irregular, post:2, topic:924133”]13.
When selecting regions I was not able to move a frequency up, only down.
[/quote]

Yes the axis points are not able to be moved around freely (like in photoshop).

Yes! Completely agree. Metasynth was actually used in the classic legendary film “The Matrix” and to this day there is no film score that sounds anything like “The Matrix”.

4

Excellent analysis irregular.
Bueno saber que “haigamos” :slightly_smiling_face: chilenos en SL

5

OK, so, this is what I meant in other post about saving selections as a library; are the saved selections only available within a specific slp job? I would really want my selection library available for use in any slp job. I will experiment further.

6

Yes! You can use saved selections and apply it to any project. Just open the project file and open any project (the project where the selections you would like to use are saved) and then open another project (not within the same project, a whole separate project under a whole new tab) and then open saved selections and that should give you the option to apply the saved selections (from previous project) to current project.

However the saved selections systems overall needs to be re-written in order for the workflow to be congruent to windows/mac applications intuitiveness.

7

awesome, thanks

seems like a global solution would be more efficient…anyway, workaround revealed! Thanks guys!

8

Jajaja buena @Nspace :chile:

I actually had misgivings about buying SpectraLayers since, aside from separating mixes and such, I had little use for it. I’m glad to say it’s been growing on me. I think additional creative applications would move it from a niche product into something everybody needs.

For instance I had this problem processing Macross:

Untitled
Untitled1567×646 153 KB

On the left is the original audio track, no music, only voice in teal and a repetitive synth in yellow. To the right you can see how much the voices had masked over the synth. It is also clearly a loop so cut and paste the pattern worked. However, I can imagine a pattern recognition tool could do this job for me, as does the stamp clone or patch tool in Adobe Photoshop.

I also noticed that, probably due to the transient nature of these sound effects, the SL modules would confuse some of them as voice.

I definitely want to know what the 3d Perspective tool is for. For reals! :smiley:

The “Unmix Transcription” was definitely a surprise. I’d say a lot more can be done with it, for instance, an export subtitles as SSA or SRT. Subtitle Edit has something similar, "Audio to Text (Whisper).

I’m curious about the “Unmix Components” and the creative posibilities of working in the frequency domain.

I wan’t this for sure! I’ll pay for it!

9

Made SL developers a “Mega Man X” spectrum of approval:

xAcceleration
xAcceleration1251×797 130 KB

Thanks for the hard work!

10

Sweet, love it, thanks!

11

Very nice result! And thank you for sharing your method.
If I understood well, the Spanish dubbed version was simply taken from a TV transmission, recording live audio?

(Otro pirilongi por acá)

12

Buena! :rofl:

Yes, that is correct, recorded on VHS many years ago from Canal 13 and then transferred over to a computer. But, for those who don’t know, Robotech had a very different music soundtrack that Macross did, so that is what made this test more interesting for me. There are moments in which you can almost hear the Robotech soundtrack peak into the test but SpectraLayers tools worked really well.
Also, the dialogue is a bit different in both shows so I edited it to reflect the Macross dialogue. But that was just because I wanted to be faithful with Macross.

13

I have a feature request for the Devs. I had a word/phrase in audio that I needed to turn into a question. I had to pitchshift up a couple of semitones and slowly raise the pitch at the end to turn it into an expressive question. This was a coarse solution but it did the job.
The feature request would be a mesh/spline distort for audio in SL. I made a mock of the tool.

Spectra_D
Spectra_D1098×642 103 KB

There is a pitchshift in a menu but it is not exactly this.

Thank you!

14

So in-other-words you mean real-time transformation based on moving the axis points(corners) on selections?

15

Yes! would be useful and fun to play around with too.

16

Hi!

I hope you still read this forum. I ended up here because I’m working on a very similar project. In my case, I’m using the Macross Blu-rays to remaster the Robotech episodes. In fact, I’ve already finished that part. I went through the first 36 episodes scene by scene, and in many cases frame by frame. It was painful, but it’s finally done. My plan is to do the same with Mospeada and then see what can be done with Southern Cross, although that’s the one with the lowest quality source.

Right now I’m trying to improve the audio, especially the original Latin American Spanish dub. I grew up watching Robotech in Chile during the '80s, so those are the voices I’m attached to.

After testing several AI models, I managed to extract most of the voices from an old TV source using melband_roformer_instvoc_duality_v1. The results are surprisingly clean in many scenes, although the model still gets confused by certain sound effects, especially jet engines and similar sounds.

My current workflow is to remove the dialogue from the English 5.1 center channel and then mix in the extracted Spanish voices. So far it sounds quite good in most episodes, although I’m still checking everything episode by episode.

I’d really like to compare my results with what you were able to achieve regarding voice extraction. I have a feeling your results might be better than what I’ve managed with AI.

At the moment I’m also preparing to rebuild the original opening and ending credits. I found a few reconstructions made by other people, but they’re all fairly low quality. I have the Robotech main title from the official soundtrack CD that was released years ago, but I’m concerned about all the original sound effects used during the opening. There are jet sounds, switches, gunfire, lasers, and even motorcycle sounds during the Mospeada sequences.

While reading this thread, I came across the Macross VOXP game. I managed to extract some sound effects from it, but unfortunately not the ones I’m looking for.

How successful were you at extracting the sound effects?

I’d be more than willing to share the Robotech restoration work I’ve done if you’re interested. I’m planning to release it publicly once I’ve rebuilt and improved everything I reasonably can.

As a side note, it seems that the narration used in the original Latin American Spanish “next episode” previews has been lost. At least I haven’t been able to find it anywhere. As a workaround, I cloned the narrator’s voice using the opening narrations from the episodes, and the results are actually pretty good.

Thanks in advance!

17

That would indeed be a very nice feature to have.

18

Hola perro! Mansa peguita que te estai mandando! I’m also a huge fan of the series. I don’t think many people know this, I’m resposable for doing the first spanish dub of Robotech that got on the internet. It’s funny how I keep running into them sometimes. Also, about a year ago I did a 16:9 test using some forms of AI. It looked very nice but that was just a test. I’ll see where I left it in case you are curious.

Right now I’m trying to improve the audio, especially the original Latin American Spanish dub. I grew up watching Robotech in Chile during the '80s, so those are the voices I’m attached to.

It’s funny you mention that. Since I did the dubs I also started learning sound production. I am also attached to it. There’s something very nice about TV broadcast multicompression.

I’m certainly interested in the project. I had heavy depression for many years and I ended up alone. I’m trying to meet people again and you seem a very interesting fellow.

Regarding the “Next Episode”, yes that might be true. When I did my dub hard drive space was scarce and I did it on a Pentium 3 so I couldn’t add to it. Let me think about what can be done about it.

Oh, audio effects. So Spectralayers was somewhat good as long as is was well defined. I had lots of problems with the Zentradi vocal effect they had. I think it was a multitap delay. That was the hardest one. By now though there is a new versión of Spectralayers that I’ve yet to experiment with so I don’t know if it would do better.

Cualquier cosa me dices no más. chao!

19

Wena bro!..parece que encontre a la persona correcta!, …a few days ago I didn’t even know this software existed, and now I find out you were directly involved in the dub I’m trying to improve. Thanks, Internet! Hahaha…

I don’t know much about sound production beyond basic Audacity and FFmpeg processing. Most of my work has been done with Python, especially testing AI models. I’m absolutely sure I could learn a lot from you, and it’s great that we share an interest in this particular TV show!

Recently I’ve been experimenting with several AI models for separating sound effects, and even some prompt-based models are doing a surprisingly decent job. That said, I don’t have enough experience with professional audio production or tools like SpectraLayers to really judge how good the results are compared to what can be achieved manually.

If you’re interested, we could share a folder and I’d be happy to upload some of my restoration results there.

Que bueno conocerte hmno!!

20

Yes sure. I’ll send over some my email so we can talk about the other stuff.

I def can teach audio production.