Printing the data in the right format

**tabstop** · 12-21-2007

Originally Posted by c_geek

These files are not that old at all. I believe from this year and may be couple of months old. And the data i am trying to read is actually a hourly array data, which gets a new time stamp after every one hour or so. each row in the array has 17 elements with the first element is the time stamp. followed by other elements.
hope this helps

Yay hourly. Okay. So a difference of 900 in the top two bytes would seem to indicate a resolution of four seconds, which is bizarre, OR that for whatever reason the the time is taking 18 bits. (A day is 86400 seconds, which requires ... seventeen bits. Close enough, I guess.) That leaves 14 bits for the date, which is still 0x1E20=7712. If we go by days, that's 21 years, one month, and 11 or so days. Seems unlikely to be a couple of months ago, more like February.

Well, let's look at bytes (remembering that the first two are "gone")
01111000100000
If we split as four bytes for months, five bytes for day, and five for year we get
0111 10001 00000 Jul-17-0 ooh.
If we split as four bytes for year, four bytes for month, and five for day (with padding at end) we get
0111 1000 10000 0 (200)7-Aug-16 which is a possibility.

Putting it all together (for "Tue Apr 19"):
001011011011010000 0111 1000 10000 0
46800 = 13:00:00 2007-Aug-16

I might have some details wrong (remember, I'm guessing about your GMT-5; did you ever print out the actual four-byte integer you had to compare with these?), but I'm starting to believe in the eighteen/fourteen split. Yay math!

edit: Ooh! idea! The reason the first one was an hour off the rest is because the date as printed was in Standard Time while the other dates land in Daylight Saving Time! Ooh! So maybe all your integers should have ended 0x1010 instead, so now we have
01000000010000 for our days. That's 11.25 years, so maybe they come in March (although why we would start counting from 1996 does not appear). Anyway, that's why the first one was off from the other three.

**matsp** · 12-21-2007

Yes, there you go, looks plausible. Not sure what 0x1010 means - maybe the date was wrong on the machine, and it was set for first of january 2001 or some such?

--
Mats

**c_geek** · 12-21-2007

Originally Posted by tabstop

Yay hourly. Okay. So a difference of 900 in the top two bytes would seem to indicate a resolution of four seconds, which is bizarre, OR that for whatever reason the the time is taking 18 bits. (A day is 86400 seconds, which requires ... seventeen bits. Close enough, I guess.) That leaves 14 bits for the date, which is still 0x1E20=7712. If we go by days, that's 21 years, one month, and 11 or so days. Seems unlikely to be a couple of months ago, more like February.

Well, let's look at bytes (remembering that the first two are "gone")
01111000100000
If we split as four bytes for months, five bytes for day, and five for year we get
0111 10001 00000 Jul-17-0 ooh.
If we split as four bytes for year, four bytes for month, and five for day (with padding at end) we get
0111 1000 10000 0 (200)7-Aug-16 which is a possibility.

Putting it all together (for "Tue Apr 19"):
001011011011010000 0111 1000 10000 0
46800 = 13:00:00 2007-Aug-16

I might have some details wrong (remember, I'm guessing about your GMT-5; did you ever print out the actual four-byte integer you had to compare with these?), but I'm starting to believe in the eighteen/fourteen split. Yay math!

edit: Ooh! idea! The reason the first one was an hour off the rest is because the date as printed was in Standard Time while the other dates land in Daylight Saving Time! Ooh! So maybe all your integers should have ended 0x1010 instead, so now we have
01000000010000 for our days. That's 11.25 years, so maybe they come in March (although why we would start counting from 1996 does not appear). Anyway, that's why the first one was off from the other three.

Hi guys
I appreciate all of the work you have done for me. As you might have noticed, i am pretty novice in this and i am right now absolutely lost. Could you tell me what am i doing wrong in printing the dates i posted early. I will really appreciate your efforts in this regard

**matsp** · 12-21-2007

Ok, so the time is (probably) a struct like this:

Code:

union timestamp {
   int tdate;
   struct {
      unsigned seconds: 18;
      unsigned year:4
      unsigned month:4;
      unsigned day: 5
   } bits;
};

The year is number of years from 2000.
Month is 1..12 [probably]
Day is 1..31
Seconds is number of seconds within the day, 0..86399, with 0 being midnight, 3600 being 01:00:00, or 1 hour past midnight, 86399 is 23:59:59, or one second away from midnight the next day.

There is of course no way to know if this is ACTUALLY correct - but what we've seen so far seems to match this.

--
Mats

**tabstop** · 12-21-2007

Originally Posted by c_geek

Hi guys
I appreciate all of the work you have done for me. As you might have noticed, i am pretty novice in this and i am right now absolutely lost. Could you tell me what am i doing wrong in printing the dates i posted early. I will really appreciate your efforts in this regard

The error, if you want to call it such, was assuming that whomever/whatever is writing out these )*(&^@($#*^%!(*&$*& timestamps does so in any kind of reasonable way. (Sorry I'm a little giddy right now.)

We were using the standard time functions from your compiler's standard library; that would be well and good, if the data was *written* that way. If they were, you would have been told such and we would never have had this thread. But (1) people don't have to use the standard libraries, especially since these particular functions don't always get a lot of play, and (2) even if they did, this particular library is implementation-defined, which means gcc doesn't have to do it the same way as VS doesn't have to do it the same way as Borland doesn't have to do it the same way as ... under the hood.

If we knew some timestamps and when they were taken, we would be able to put some puzzle pieces together. It's not programming, so much as meta-programming, or maybe reverse-engineering (given the outputs and the inputs, figure out a function that does so).

Now you have to do the programming; we've figured out what the data means, you have to write the code to process it.

**matsp** · 12-21-2007

Whilst I agree that the approach of "every programmer invents his own way to store timestamps" is a bit stupid, we don't know the full story.

The data may have come from an embedded system or some such, that hasn't got a full set of standard library functions in the first place!

And it's still a guess - unless we have a RANGE of known dates [preferrably over a reasonable time-period, severam months, and at least passing ONE new year] with known encoding values, this is probably as far as we're going to get.

Do you know if 16 Aug 2007 is ABOUT right for your timestamps? Does the file itself have a "creation date" that matches the dates in the file?

--
Mats

**c_geek** · 12-24-2007

Originally Posted by matsp

Whilst I agree that the approach of "every programmer invents his own way to store timestamps" is a bit stupid, we don't know the full story.

The data may have come from an embedded system or some such, that hasn't got a full set of standard library functions in the first place!

And it's still a guess - unless we have a RANGE of known dates [preferrably over a reasonable time-period, severam months, and at least passing ONE new year] with known encoding values, this is probably as far as we're going to get.

Do you know if 16 Aug 2007 is ABOUT right for your timestamps? Does the file itself have a "creation date" that matches the dates in the file?

--
Mats

16 Aug 2007 looks right to me as those files (where i am reading from) can be that old. I have tried to print the first 4 bytes of the first three rows in the file, here are they:

Code:

Row 1 date/Ts first byte is       :  0 (0x0)
Row 1 date/Ts second byte is  :  0 (0x0)
Row 1 date/Ts third byte is      :  0 (0x0)
Row 1 date/Ts fourth byte is    :  0 (0x0)

Row 2 date/Ts first byte is       :  44 (0x2C)
Row 2 date/Ts second byte is  :  32 (0x20)
Row 2 date/Ts third byte is      :  28 (0x1C)
Row 2 date/Ts fourth byte is    :  170 (0xAA)

Row 3 date/Ts first byte is       :  156 (0x9C)
Row 3 date/Ts second byte is  : 24   (0x18)
Row 3 date/Ts third byte is      :  52  (0x34)
Row 3 date/Ts fourth byte is    :  243 (0xF3)

Also matt you made a union and then struct inside a union, i think the whole idea behind is to use this format to print the date/Ts instead of standard ctime() function? is that right?

**matsp** · 12-24-2007

Originally Posted by c_geek

Also matt you made a union and then struct inside a union, i think the whole idea behind is to use this format to print the date/Ts instead of standard ctime() function? is that right?

Yes, that's the idea.

I would suggest you try that out and see what you get.

If it looks sensible, go for it. If it comes up all jumbled, could you print about 10 or so of the date/time fields as a 32-bit integer with the %08x format specifier, rather than as individual bytes?

--
Mats

**c_geek** · 12-24-2007

Originally Posted by matsp

Yes, that's the idea.

I would suggest you try that out and see what you get.

If it looks sensible, go for it. If it comes up all jumbled, could you print about 10 or so of the date/time fields as a 32-bit integer with the %08x format specifier, rather than as individual bytes?

--
Mats

Here are about 12 date/time prints in 32 bits long variable t:

Code:

Row Date is:00000000
Row Date is:1fa42c33
Row Date is:1c202c33

Row Date is:189c2c33
Row Date is:15182c33
Row Date is:11942c33

Row Date is:0e102c33
Row Date is:0a8c2c33
Row Date is:07082c33

Row Date is:03842c33
Row Date is:00002c33
Row Date is:50dc2c32

Also i guess i need to make tdate variabel in your defined union as long instead of int? is that right

**matsp** · 12-24-2007

Originally Posted by c_geek

Also i guess i need to make tdate variabel in your defined union as long instead of int? is that right

Well, assuming you are using a 32-bit compiler [e.g. gcc, Visual Studio or a Unix cc compiler], rather than a 16-bit (e.g. Turbo C), then an int should be 32 bits. Long will not make it differente, but if you ever move to a 64-bit architecture, long MAY a 64-bit value, which won't help.

--
Mats

**c_geek** · 12-24-2007

Originally Posted by matsp

Well, assuming you are using a 32-bit compiler [e.g. gcc, Visual Studio or a Unix cc compiler], rather than a 16-bit (e.g. Turbo C), then an int should be 32 bits. Long will not make it differente, but if you ever move to a 64-bit architecture, long MAY a 64-bit value, which won't help.

--
Mats

Does the prints of the date/time stamp make sense in my previous reply?

**matsp** · 12-24-2007

Originally Posted by c_geek

Does the prints of the date/time stamp make sense in my previous reply?

I'm looking at that at this very moment. I'm also watching a 1-year old and a 3-year old, which can be a bit of a burden.

--
Mats

**matsp** · 12-24-2007

Code:

2007-03-12 18:00:01
2007-03-12 16:00:01
2007-03-12 14:00:01
2007-03-12 12:00:01
2007-03-12 10:00:01
2007-03-12 08:00:01
2007-03-12 06:00:01
2007-03-12 04:00:01
2007-03-12 02:00:01
2007-03-12 00:00:01
2006-03-12 09:35:29

Does that look about right, or not quite?

It seems a bit backwards that it starts with the "later time", but it doesn't make much sense any other way either.

--
Mats

**c_geek** · 12-24-2007

Originally Posted by matsp

Code:

2007-03-12 18:00:01
2007-03-12 16:00:01
2007-03-12 14:00:01
2007-03-12 12:00:01
2007-03-12 10:00:01
2007-03-12 08:00:01
2007-03-12 06:00:01
2007-03-12 04:00:01
2007-03-12 02:00:01
2007-03-12 00:00:01
2006-03-12 09:35:29

Does that look about right, or not quite?

It seems a bit backwards that it starts with the "later time", but it doesn't make much sense any other way either.

--
Mats

It looks pretty close to me,except the last entry "2006-03-12 09:35:29 ( i am assuming that it is in the format of yyyy-dd-mm hr:min:sec) i was expecting to have the day changed not the year.
Also how did you print out this in this format? I will appreciate your time and efforts in this regards. Thanks

**matsp** · 12-24-2007

Some slight modification to the format:

Code:

#include <stdio.h>

int array[] = 
{0x00000000,
 0x1fa42c33,
 0x1c202c33,

 0x189c2c33,
 0x15182c33,
 0x11942c33,

 0x0e102c33,
 0x0a8c2c33,
 0x07082c33,

 0x03842c33,
 0x00002c33,
 0x50dc2c32,
};


union timestamp {
   int tdate;
   struct {
      unsigned day:5;
      unsigned month:4;
      unsigned year: 4;
	  unsigned :1;
      unsigned seconds: 17;
   } bits;
};

void swap(unsigned char &a, unsigned char &b)
{
	unsigned char t;
	t = a;
	a = b;
	b = t;
}

void byteswap(int &t)
{
	unsigned char *x = (unsigned char *)&t;
    swap(x[0], x[3]);
	swap(x[1], x[2]);
}

int main(void)
{
	int i;
	timestamp tt;
	for(i = 1; i < 12; i++)
	{
		tt.tdate = array[i];
		printf("%04d-%02d-%02d %02d:%02d:%02d\n", tt.bits.year+2001, tt.bits.month, tt.bits.day, 
				tt.bits.seconds / 3600, (tt.bits.seconds % 3600) / 60, tt.bits.seconds % 60);
	}
	return 0;
}

Gives this output:

Code:

2007-01-19 09:00:00
2007-01-19 08:00:00
2007-01-19 07:00:00
2007-01-19 06:00:00
2007-01-19 05:00:00
2007-01-19 04:00:00
2007-01-19 03:00:00
2007-01-19 02:00:00
2007-01-19 01:00:00
2007-01-19 00:00:00
2007-01-18 23:00:00