Thread: Info Theory

  1. #1
    Registered User
    Join Date
    May 2006
    Posts
    12

    Info Theory

    We have 14 discrete symbols s1, s2, … s14. Each symbol haves the probability p1, p2, …, p14 respectively, to be transmitted. Is the average transmitted information in bits (p1*log2(1) + p2*log2(2). … , + p14*log2(14) ) /14 or (p1*log2(14) + p2*log2(14). … , + p14*log2(14) ) /14 or something else? Is the information for each symbol in bits log2(14) or something else? (I am not a student, and sorry its not a subject of C).
    Thanks.

  2. #2
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    So if it's not a C topic, why did you feel the need to post it on the C forum, then fein apology?


    Quzah.
    Hope is the first step on the road to disappointment.

  3. #3
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,656
    Moved to tech.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  4. #4
    geek SilentStrike's Avatar
    Join Date
    Aug 2001
    Location
    NJ
    Posts
    1,141
    Quote Originally Posted by vangmor
    We have 14 discrete symbols s1, s2, … s14. Each symbol haves the probability p1, p2, …, p14 respectively, to be transmitted. Is the average transmitted information in bits (p1*log2(1) + p2*log2(2). … , + p14*log2(14) ) /14 or (p1*log2(14) + p2*log2(14). … , + p14*log2(14) ) /14 or something else? Is the information for each symbol in bits log2(14) or something else? (I am not a student, and sorry its not a subject of C).
    Thanks.
    The entropy is p1 log2 (1/ pi) + p2 log2 (1 / p2) + p3 log2 (1 / p3) .. p14 log (1 / p14). The entropy is maximized when all the probabilities are equally likely, so all are 1/14, in which case the entropy is log2(14).
    Prove you can code in C++ or C# at TopCoder, referrer rrenaud
    Read my livejournal

  5. #5
    Registered User
    Join Date
    May 2006
    Posts
    12
    Thanks SilentStrike!

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. help displaying info from structure
    By kisiellll in forum C Programming
    Replies: 6
    Last Post: 04-04-2009, 12:51 PM
  2. Question about getting an info class from another Form
    By Joelito in forum C# Programming
    Replies: 0
    Last Post: 10-16-2006, 01:02 PM
  3. MMO Theory, How to make Gameloop & Sockets both work?
    By Zeusbwr in forum Game Programming
    Replies: 3
    Last Post: 08-01-2005, 12:29 PM
  4. Help doing an e-mail program in c...
    By Tyler_Durden in forum C Programming
    Replies: 88
    Last Post: 01-02-2005, 03:12 PM
  5. Replies: 3
    Last Post: 12-06-2001, 05:30 AM