Nobody will be able to tell you exactly what's wrong (I would suspect you have a great number of problems with your code), since you don't provide us much useful info. We have no part number for the LCD. We have no schematic to see how the LCD is wired to the uC. We don't have any of the LCD code, so we don't know if you're talking to it correctly. There are timing issues to consider when writing to the LCD (you may need small delays between setting enable pins low and high). You can't safely check which one was triggered first with just if statements, you will need some sort of loop.

You will need to approach this in several discreet steps. The first thing I recommend in your case is to get the LCD working. This takes a bit of time, but you need to do it anyhow, so get it done first and use the LCD for debugging purposes. Then, make sure you can read P3.0 and P3.1 correctly. Simply display their values on the LCD, 1 for high (beam intact), 0 for low (beam broken). Wire them up to some simple switches for testing (note that without a debounce circuit, you may get some transient bogus data on your LCD, but they should settle out quickly). Then work on the logic for detecting in which order the beams were interrupted and whether somebody entered or exited the library.

The general idea is this:
  1. Wait for one of the two beams to go low
  2. Check which one is low
  3. Then, wait for the other beam to go low
  4. Record an entry or exit as appropriate
  5. Wait for both beams to go high again
  6. Repeat

The waits should be some sort of loop, the repeat could be handled with a while(1).