malloc() and read() limits

This is a discussion on malloc() and read() limits within the Linux Programming forums, part of the Platform Specific Boards category; I have a code segment which reads: <..> char *maps; <..> maps = (char *)malloc(65536); if (maps == NULL) return ...

  1. #1
    Registered User
    Join Date
    Feb 2006
    Posts
    3

    malloc() and read() limits

    I have a code segment which reads:

    <..>
    char *maps;
    <..>
    maps = (char *)malloc(65536);
    if (maps == NULL)
    return -1;
    size = read (fd, maps, 65536);
    <..>

    This code runs on Linux kernel 2.4 with glibc2.3. On certain systems of this platform the value of size is always capped at 4096 bytes. The actual file pointed to by fd is larger than that.

    The same code runs without this 4096 appearing as a limit.

    What should I be looking at?

    I do not have access to the symptomatic systems but I instrumented the program to get some ideas as to whether it is a system-wide limit or some such thing. I got the following:

    RLIMIT_AS(cur,max): <-1, -1>
    RLIMIT_CORE(cur,max): <-1, -1>
    RLIMIT_CPU(cur,max): <-1, -1>
    RLIMIT_DATA(cur,max): <-1, -1>
    RLIMIT_FSIZE(cur,max): <-1, -1>
    RLIMIT_MEMLOCK(cur,max): <4096, 0>
    RLIMIT_NOFILE(cur,max): <1024, 0>
    RLIMIT_NPROC(cur,max): <4084, 0>
    RLIMIT_STACK(cur,max): <10485760, 0>
    RLIMIT_RSS(cur,max): <-1, -1>

    What else should I be looking at? Is this a malloc related limit or it is related to the read() call?

    Help would really be appreciated by this novice.

  2. #2
    Registered User
    Join Date
    Feb 2006
    Posts
    3
    Just one more thing, the file being processes resides in the /proc file system.

    Cheers

  3. #3
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,498
    I personally think it's a reading the manual issue

    It is not an error if this number is smaller than the
    number of bytes requested; this may happen for example because fewer bytes are actually
    available right now (maybe because we were close to end-of-file, or because we are reading
    from a pipe, or from a terminal), or because read() was interrupted by a signal.

  4. #4
    Registered User
    Join Date
    Feb 2006
    Posts
    3
    The number of bytes read, however, is always a nice 4096 bytes on three different but similarly configured systems. These systems have alot of memory but exhibit this behavior whether the systems are under load or not.

    I do not know what to consider. I've attempted a temporary workaround of reading the file in blocks of 2K and that works. But I'm curious as to what it is that's causing this. I imagine there would be no IO issues with multiple reads towards the /proc file system, right?

  5. #5
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,498
    Since /proc isn't a real file, who knows what to expect in general.

    IMO, it's a bug in your code to assume that asking for x bytes and assuming that you'll always get x bytes back.

    What is so hard about looking at the return result, and calling read in a loop until your buffer is full or there is no more data to read?

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. memory leaks
    By TehOne in forum C Programming
    Replies: 4
    Last Post: 10-10-2008, 09:33 PM
  2. Wierd Malloc Problem
    By mohankarthik in forum C Programming
    Replies: 11
    Last Post: 09-17-2008, 02:14 PM
  3. Replies: 8
    Last Post: 03-10-2008, 11:57 AM
  4. Problem with malloc() and sorting words from text file
    By goron350 in forum C Programming
    Replies: 11
    Last Post: 11-30-2004, 09:01 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21