Thread: Know Youtube video size in advance

Here in Angola I'm under a restricted internet monthly plan (boy, I miss Europe!), so I'm usually very picky about what I watch or don't watch on Youtube. Especially because a 2 GB plan costs 4,500 Kwanzas here, the equivalent to 45 dollars... yes, that's the correct math.

So, I really would love to find a way of having a glimpse at the size of a Youtube video I plan to watch in any of the available resolutions. I'm looking into a way of getting that information from the web, or off of Youtube itself so that I can write a script myself.

Thanks.

Send a request for get_video_info:
https://www.youtube.com/get_video_info?video_id=v&ps= default&eurl=&gl=US&hl=en

Of course replace v by whatever the video id of the video is (ie the string at the end of video page's url)

You'll get a text file with a set of ascii encoded key-value pairs. Like this:

Code:

 key1=value1&key2=value2...&keyN=valueN

Parse it and unhex each value (ie many values have %hh characters which must be decoded to the hex equivalent of hh. ex: %41 in a string would decode to 'A'. You may need to do multiple passes before there are no more %hh sequences)

Then look for the keys url_encoded_fmt_stream_map and adaptive_fmts, they'll have URLs for each availabe resolution of video, the urls are sepearated by ; marks ...it's a pretty big string.

Also remember that it is IP dependent and time limited, something to note if you have dynamic IPs.

When you put the URL in the browser address bar, it'll come as a normal download which will show the size of the video.

This unfortunately doesent work for all videos. Wait I'll give you an example...

here's the C code I used to decode the values in the key-value pairs:

Code:

unsigned char toHex(char c) {
	if(c>='a'&&c<='f') return c-87;
	else if(c>='A'&&c<='F') return c-55;
	else if(isdigit(c)) return c-48;
}

unsigned char hex_readByte(char *str) {
	unsigned char v1=toHex(*str),v2=toHex(str[1]);
	return ((v1<<4)|v2);
}

// single step of unhex
int unhexi(char *str) {
    char *src=str,*dest=str;
    char now;
    int c=0;

    while(1) {
        now=*src;

        if(now==0) break;
        else if(now=='%') {
            src++;
            *dest++=hex_readByte(src);
            src+=2;
            c++;
        } else *dest++=*src++;
    }

    *dest=0;
    return c;
}

void unhex(char *str) {
    while(unhexi(str));
}

Example value of url_encoded_fmt_stream_map:

Code:

type=video/mp4;+codecs="avc1.64001F,+mp4a.40.2"&url=https://r17---sn-cvh7zn7z.googlevideo.com/videoplayback?mm=31&source=youtube&fexp=900718,907263,923374,924637,927622,931361,936121,9405703,9405890,941004,943917,947209,947225,948124,952302,952605,952901,955301,957103,957105,957201,959701&dur=87.771&key=yt5&ip=117.232.42.95&expire=1421708983&ratebypass=yes&ipbits=0&signature=7F8D40FD7D6CD1D28ABAD435C6884A5659B1DEAF.D7D80DDE1BE45FF2CA38EA4058C330222BFCBBDB&itag=22&upn=BQOdGoWT_ew&sver=3&requiressl=yes&sparams=dur,id,ip,ipbits,itag,mm,ms,mv,pl,ratebypass,requiressl,source,upn,expire&mt=1421687305&pl=21&mv=m&id=o-AOKQbvjPqkIjxN-eI1-SvIphjWzbNt6LFWrymLPrlFkh&ms=au&itag=22&fallback_host=tc.v5.cache6.googlevideo.com&quality=hd720,type=video/webm;+codecs="vp8.0,+vorbis"&url=https://r17---sn-cvh7zn7z.googlevideo.com/videoplayback?mm=31&source=youtube&fexp=900718,907263,923374,924637,927622,931361,936121,9405703,9405890,941004,943917,947209,947225,948124,952302,952605,952901,955301,957103,957105,957201,959701&dur=0.000&key=yt5&ip=117.232.42.95&expire=1421708983&ratebypass=yes&ipbits=0&signature=AE0EC16AEF62EB8D4D7234CBADC1DDFE8335786B.E3008993C40768A0518C4320154AFAEB1118AE0C&itag=43&upn=BQOdGoWT_ew&sver=3&requiressl=yes&sparams=dur,id,ip,ipbits,itag,mm,ms,mv,pl,ratebypass,requiressl,source,upn,expire&mt=1421687305&pl=21&mv=m&id=o-AOKQbvjPqkIjxN-eI1-SvIphjWzbNt6LFWrymLPrlFkh&ms=au&itag=43&fallback_host=tc.v6.cache4.googlevideo.com&quality=medium,type=video/mp4;+codecs="avc1.42001E,+mp4a.40.2"&url=https://r17---sn-cvh7zn7z.googlevideo.com/videoplayback?mm=31&source=youtube&fexp=900718,907263,923374,924637,927622,931361,936121,9405703,9405890,941004,943917,947209,947225,948124,952302,952605,952901,955301,957103,957105,957201,959701&dur=87.771&key=yt5&ip=117.232.42.95&expire=1421708983&ratebypass=yes&ipbits=0&signature=A30806FD9C43BF49B27A99E884F41C22BCCE005C.7A10A56CE977F1D4F9DB69B9B0EB664667FB68C0&itag=18&upn=BQOdGoWT_ew&sver=3&requiressl=yes&sparams=dur,id,ip,ipbits,itag,mm,ms,mv,pl,ratebypass,requiressl,source,upn,expire&mt=1421687305&pl=21&mv=m&id=o-AOKQbvjPqkIjxN-eI1-SvIphjWzbNt6LFWrymLPrlFkh&ms=au&itag=18&fallback_host=tc.v19.cache5.googlevideo.com&quality=medium,type=video/x-flv&url=https://r17---sn-cvh7zn7z.googlevideo.com/videoplayback?mm=31&source=youtube&fexp=900718,907263,923374,924637,927622,931361,936121,9405703,9405890,941004,943917,947209,947225,948124,952302,952605,952901,955301,957103,957105,957201,959701&dur=87.771&key=yt5&ip=117.232.42.95&expire=1421708983&ipbits=0&signature=3733D7F2E831D3E1CDB13CB5342D78471BCAD892.43238C555853F3FFE0221B9A5601BA9285D2D4A6&itag=5&upn=BQOdGoWT_ew&sver=3&requiressl=yes&sparams=dur,id,ip,ipbits,itag,mm,ms,mv,pl,requiressl,source,upn,expire&mt=1421687305&pl=21&mv=m&id=o-AOKQbvjPqkIjxN-eI1-SvIphjWzbNt6LFWrymLPrlFkh&ms=au&itag=5&fallback_host=tc.v19.cache3.googlevideo.com&quality=small,type=video/3gpp;+codecs="mp4v.20.3,+mp4a.40.2"&url=https://r17---sn-cvh7zn7z.googlevideo.com/videoplayback?mm=31&source=youtube&fexp=900718,907263,923374,924637,927622,931361,936121,9405703,9405890,941004,943917,947209,947225,948124,952302,952605,952901,955301,957103,957105,957201,959701&dur=87.910&key=yt5&ip=117.232.42.95&expire=1421708983&ipbits=0&signature=F6DAF39935B004DFA6899EFF39A0A4EF77FEFFF3.BC99E5A90E5993F6F96BDFAEAA9BDD25BD8EA33E&itag=36&upn=BQOdGoWT_ew&sver=3&requiressl=yes&sparams=dur,id,ip,ipbits,itag,mm,ms,mv,pl,requiressl,source,upn,expire&mt=1421687305&pl=21&mv=m&id=o-AOKQbvjPqkIjxN-eI1-SvIphjWzbNt6LFWrymLPrlFkh&ms=au&itag=36&fallback_host=tc.v3.cache3.googlevideo.com&quality=small,type=video/3gpp;+codecs="mp4v.20.3,+mp4a.40.2"&url=https://r17---sn-cvh7zn7z.googlevideo.com/videoplayback?mm=31&source=youtube&fexp=900718,907263,923374,924637,927622,931361,936121,9405703,9405890,941004,943917,947209,947225,948124,952302,952605,952901,955301,957103,957105,957201,959701&dur=88.050&key=yt5&ip=117.232.42.95&expire=1421708983&ipbits=0&signature=EDE131BA8B9BE64AF7D0D284D801D8428D059DA5.3EF0672535EA964D196298B18ABE28A76BCDC5CA&itag=17&upn=BQOdGoWT_ew&sver=3&requiressl=yes&sparams=dur,id,ip,ipbits,itag,mm,ms,mv,pl,requiressl,source,upn,expire&mt=1421687305&pl=21&mv=m&id=o-AOKQbvjPqkIjxN-eI1-SvIphjWzbNt6LFWrymLPrlFkh&ms=au&itag=17&fallback_host=tc.v12.cache3.googlevideo.com&quality=small

Copying the text from &url= to ; for the first url for example we get:

Code:

https://r17---sn-cvh7zn7z.googlevideo.com/videoplayback?mm=31&source=youtube&fexp=900718,907263,923374,924637,927622,931361,936121,9405703,9405890,941004,943917,947209,947225,948124,952302,952605,952901,955301,957103,957105,957201,959701&dur=87.771&key=yt5&ip=117.232.42.95&expire=1421708983&ratebypass=yes&ipbits=0&signature=7F8D40FD7D6CD1D28ABAD435C6884A5659B1DEAF.D7D80DDE1BE45FF2CA38EA4058C330222BFCBBDB&itag=22&upn=BQOdGoWT_ew&sver=3&requiressl=yes&sparams=dur,id,ip,ipbits,itag,mm,ms,mv,pl,ratebypass,requiressl,source,upn,expire&mt=1421687305&pl=21&mv=m&id=o-AOKQbvjPqkIjxN-eI1-SvIphjWzbNt6LFWrymLPrlFkh&ms=au&itag=22&fallback_host=tc.v5.cache6.googlevideo.com&quality=hd720,type=video/webm

Naturally this wont work cuz this is IP dependent and I was using an old saved get_video_info file. But this will work in some cases when done on a freshly downloaded video_info file.

Which is to say once you get a working URL to the raw video file, it's just like a normal HTTP download so you easily know the file size beforehand.

Originally Posted by Yarin

Wowza. Have you retried Firefox/Chrome for lynx as your daily browser yet?

Stuff like Steam or my daily dose of TED videos, all are off limits here. I have maybe 20 games that I bought during Steam sales that I haven't installed yet. Some months I do go on about to spend 300 USD on my internet connection. But that's when I need to sync with my seedbox (at which time I also take the opportunity to update my windows and linux installations and do other heavy download stuff). But the rest of the time, since I got here, my internet usage has been very limited.

Jackass, thank you for the info. I will build something out of this. I'll use Python since I'm interested in putting my recent learning of it to practice.

No problem xD

Just remember my earlier warning, this method doesent work in some videos (that is ends in a 501 access denied for them).

Any idea why? Private videos, youtube bug?

Probably youtube just keeps updating it's tricks to avoid people from writing downloaders. They've changed their methods many times in the past too.

There are advanced techniques used by youtube downloader programs that I'm ignorant of, they do tricks like parse the javascript in the youtube page and auto-guess the new algorithms on it's basis and adapt themselves to those algorithms. So these programs dont fail even when youtube has changed their methods.

O_o

You say you are going to use the Python language?

Download the "youtube-dl" application which can be used as a Python library.

You can grab the size very easily while letting other people worry about the details Google changes things.

You could even conditionally download the video if the size works for you.

Soma

Originally Posted by phantomotap

youtube-dl

Oh that is exactly what I needed!

I went through the youtube-dl options (bloody thing is big on options) and found exactly what I needed. The -F and -f flags are all that is required. Here's the output I got on a random test video:



The video+audio formats don't display file size unfortunately. But that's a minor snag. I can always download the separated audio file above and combine it with the video with VidJoin or other similar tool.

Thanks very much. This is all I need.

Hmm... a related question:

I download video format 160 and audio format 139 above. Then I joined both with KeepVid VideoJoin. When I played the movie not long the audio went increasingly out of sync with the video.

I don't know much about video and audio processing. Is there any way I can gauge at the list above and calculate the correct audio file for the video's FPS I am downloading? Or alternatively, can I download just any audio file and then edit some aspect of it on an audio editor so I can get it to play in sync?

I download video format 160 and audio format 139 above. Then I joined both with KeepVid VideoJoin. When I played the movie not long the audio went increasingly out of sync with the video.

O_o

I don't have enough information to truly speculate, but I'd say "KeepVid VideoJoin" is part of your problem. Your application may be stretching the actual audio to fit the video. I don't get an option for that audio stream to take a look at the files, but I looked at what was listed, and the audio file has a couple of seconds of silence at the end. I used "ffmpeg" to join the files I was able to get and the result seemed fine, but that doesn't count for much because I couldn't get that audio file.

I'd suggest you open both files in separate instances of your preferred media player. You may need to take a few tries, but you should learn something. If you can get the files started with synchronization you can at least if the files don't match or if the application might be doing something wonky.

Or alternatively, can I download just any audio file and then edit some aspect of it on an audio editor so I can get it to play in sync?

I'd suggest trying the 140 file with "KeepVid VideoJoin" before you make your next move. (You would try this after the earlier suggestion.) I understand the limited bandwidth, but I can't really help further if I don't have more information.

*shrug*

Not to say that someone else will not come along with a solution.

Soma

Originally Posted by the_jackass

Send a request for get_video_info:
https://www.youtube.com/get_video_info?video_id=v&ps= default&eurl=&gl=US&hl=en

Of course replace v by whatever the video id of the video is (ie the string at the end of video page's url)

You'll get a text file with a set of ascii encoded key-value pairs. Like this:

Code:

  key1=value1&key2=value2...&keyN=valueN

Parse it and unhex each value (ie many values have %hh characters which must be decoded to the hex equivalent of hh. ex: %41 in a string would decode to 'A'. You may need to do multiple passes before there are no more %hh sequences)

Then look for the keys url_encoded_fmt_stream_map and adaptive_fmts, they'll have URLs for each availabe resolution of video, the urls are sepearated by ; marks ...it's a pretty big string.

Also remember that it is IP dependent and time limited, something to note if you have dynamic IPs.

When you put the URL in the browser address bar, it'll come as a normal download which will show the size of the video.

This unfortunately doesent work for all videos. Wait I'll give you an example...

Thanks, this was really fascinating to hear the_jackass (wow, your name fits into a sentence all the wrong ways lol).

I was wondering if the key/value pairs are formed in hierarchical structure? I tried this trick, and ran the code to swap the hex values, and noticed that (for instance) the 'url_encoded_fmt_stream_map' appears to contain multiple properties of its own.

I'm also curious, when you put the link for 'url_encoded_fmt_stream_map' in the browser and get a download prompt, what is your request representing to the server? Does it think you are a video player, a clients web browser, or something else?