AudioScope: in-the-wild on-screen sound enhancement

Input video

Video clip from "Grand symphony" by saimad, license: CC-BY-SA 2.0, with additional background audio clip from "Mckenzie on the scene" by pescatello, license: CC-BY 2.0.

Audio clip from "Grand symphony" by saimad, license: CC-BY-SA 2.0, with additional background audio clip from "Mckenzie on the scene" by pescatello, license: CC-BY 2.0.

Audio clip from "Grand symphony" by saimad, license: CC-BY-SA 2.0.

Audio clip from "Mckenzie on the scene" by pescatello, license: CC-BY 2.0.


On-screen estimate from the input video and corresponding attention map

Video clip from "Grand symphony" by saimad with modified audio and overlaid attention map, license: CC-BY-SA 2.0.





Separated sources with input video and corresponding attention maps

Video clip from "Grand symphony" by saimad with modified audio and overlaid attention map, license: CC-BY-SA 2.0.


Video clip from "Grand symphony" by saimad with modified audio and overlaid attention map, license: CC-BY-SA 2.0.


Video clip from "Grand symphony" by saimad with modified audio and overlaid attention map, license: CC-BY-SA 2.0.


Video clip from "Grand symphony" by saimad with modified audio and overlaid attention map, license: CC-BY-SA 2.0.



Attention maps over time for on-screen estimate

Still frames from "Grand symphony" by saimad with overlaid attention maps, license: CC-BY-SA 2.0.