Need to know parameter format for a Call function?

[Will1]

phantomotap posted the results along with the shell script used to obtain those results. That is hardly "a bunch of bull".

I didn't see his results. what post did he put them in?

A quick check of the results posted by phantomotap shows that 10000 rows were added in a fraction of a second. Of course, there's the big elephant in the room as stated in post #65: these comparisons are only useful in making/debunking specious claims since there are so many factors that are left unaccounted.

I changed the add records count to 25,000 and the result was that it took about 1/3 of a second or so, I'd estimate, and took up 12.87 Megabytes of disk space..

SQL is not a relational database system. I explained this in post #13. You might compare ADAM with relational database systems in general, or with a specific relational database system (e.g., DB2, SQLite), or with some other database system (e.g., Berkeley DB, Kyoto Cabinet). In general, relational database systems use some dialect of SQL, but "SQL data base" is not the correct term, especially considering that it is often not necessarily to write SQL to access a relational database from within a program.

The only thing that you can compare ADAM with is IBM's VSAM which, as I said in a previous post, I may have designed. Google it!

That does not matter since an ad hominem argument is a logical fallacy.

WOW, impressive, you know a little Latin!

Anyway, "Who is John Galt?"

[phantomotap]

Does this guy remind anybody else of CommonTater?

O_o

If you reference what I imagine, the similarity is no stronger for CommonTater than many of the other "old hats" that refuse to keep their knowledge relevant.

You'll see such in just about every field.

[Edit]
There is actually a good episode of "Scrubs" taking up the issue.

Well, not so good if you are complaining about the attitude and not the underlying cause.
[/Edit]

Soma

[Matticus]

If you reference what I imagine, the similarity is no stronger for CommonTater than many of the other "old hats" that refuse to keep their knowledge relevant.

For me, the comparison was funny in the context of members known on these forums - and was based on some (admittedly humorous) personality similarities. But definitely not an absolute comparison.

Like Tater: Uses advanced age to justify authority
Unlike Tater: Doesn't see the Windows OS as the greatest
Like Tater: Makes substantial claims based on personal experience without providing supporting data
Unlike Tater: Not banned

[Will1]

Unlike Tater: Not banned

Banning someone is in violation of the Supreme Law, the First Amendment to our Constitution guarantees the right to freedom of speech. Banning Tater is denying him his right to speak.

I probably will be banned before long because you brain washed children don't want to hear the truth like the following:

Comparing a PC to a mainframe is like comparing a Cesna 150 to a Boeing 747 jetliner. Sure you can round up a few hundred Cesna 150s and carry as many pasengers as a 747 jetliner can but what a mess you'd have like rounding up a few hundred PCs to have as much computer power as an IBM system 390 has. They still couldn't do what a system 390 can do. Like a whole herd of Cesna 150s couldn't carry a space shuttle across the country like a 747 did.

Anyway, "Who is John Galt?"

[stahta01]

Banning someone is in violation of the Supreme Law, the First Amendment to our Constitution guarantees the right to freedom of speech. Banning Tater is denying him his right to speak.

FYI: This website is NOT being ran by the USA Federal Government!
I hope you know more about programming than you know about the USA constitution.

Tim S.

[phantomotap]

I didn't see his results. what post did he put them in?

O_o

Oh. Now I see where the CommonTater comparison was born...

Anyway, "Who is John Galt?"

Are you really so far removed from modern technology that you feel that put upon in this thread? Really?

What? Are we now supposed to feel sorrow for you and buy your nonsense as Yog Sothoth's own gospel?

Wait. I can also make allusions to literature! That must I am the co-developer of "VSAM".

Time to get paid...

dolla dolla bill yall

[laserlight]

I didn't see his results. what post did he put them in?

Post #69. Refer to the output of two runs (with different arguments) of the shell script posted.

The only thing that you can compare ADAM with is IBM's VSAM

You forgot to qualify that with "with respect to speed", but even then that remains to be seen since the timings that you quoted are easily achievable.

WOW, impressive, you know a little Latin!

I happen to know appropriate terms in English inherited from Latin, and because I assist at Mass in Latin, I happen to understand quite a few set phrases in Latin. That does not equate to knowing Latin, no more than someone able to parrot out memorised C code knows C. So, sorry, bad guess

Banning someone is in violation of the Supreme Law, the First Amendment to our Constitution guarantees the right to freedom of speech.

You misunderstand the nature of freedom of speech in a privately owned forum. (Besides making the mistake of assuming that we have a shared Constitution.)

[Will1]

Post #69. Refer to the output of two runs (with different arguments) of the shell script posted.

I added 5,000 (Now 25,000) records to an ADAM file and suggested a test of adding that many rows to an SQL (System Querry Language) data base and have yet to hear any results of how long it takes.

You forgot to qualify that with "with respect to speed", but even then that remains to be seen since the timings that you quoted are easily achievable.

Yeah and also I meant with respect ro KSDS, RRDS, and ESDS.

You misunderstand the nature of freedom of speech in a privately owned forum. (Besides making the mistake of assuming that we have a shared Constitution.)

Yeah, the politically motivated news media of this country often practices censorship of opposing political views and can do it because they are privately owned.

OK, You're not from the USA. The Constitution of the United States and the Bill of Rights are the most inspired documentation of how a civilization should be composed ever written. Those who embrace "The Communist Manifesto" want to take the right of the individual away so they force people to do their bidding. That is also true of Fascists. Here in the USA you are free to spout any kind of philosophy and not be censored. Anyway, "Who is John Galt?"

[laserlight]

I added 5,000 (Now 25,000) records to an ADAM file and suggested a test of adding that many rows to an SQL (System Querry Language) data base and have yet to hear any results of how long it takes.

You appear to have yet to read post #69 in its entirety. Anyway, after modifying phantomotap's shell script to wrap the insertions in a single transaction, I get about 0.9s on average for 25000 records, if we just count wall clock time. It has been stressed to you so many times: this doesn't really mean anything; it is fairly useless as a benchmark, but there you have it: SQLite3 "can add thousands of records to a file in a fraction of a second".

Anyway, "Who is John Galt?"

Sorry, not an Ayn Rand fan, so I'm not impressed.

[Will1]

You appear to have yet to read post #69 in its entirety.

Een the DOS version of ADAM files will hold up to 4 Gigabytes of data. What I was looking for is the speed of SQL data bases adding Rows and the space SQL uses to store rows of data.

Anyway, after modifying phantomotap's shell script to wrap the insertions in a single transaction, I get about 0.9s on average for 25000 records, if we just count wall clock time. It has been stressed to you so many times: this doesn't really mean anything; it is fairly useless as a benchmark, but there you have it: SQLite3 "can add thousands of records to a file in a fraction of a second".

I changed the number of records to be added to 50,000 and ADAM took nearly a second to do it. Still twice as fast as you were able to achieve with adding rows to an SQL data base. It used less than 26 Megabytes of disk space. How much space does an SQL data base with 512 byte rows take up on the hard drive?

We've wandered all over the place here so lets get back to the reason I started this thread. What I was really looking for is the expertise to help me create a server that allows clients to create and access an ADAM file. Since no-one seems to be interested in real software development I probably won't be posting much anymore. What I've done so far seems to be a waste of time.

Sorry, not an Ayn Rand fan, so I'm not impressed.

I think you should read "Atlas Shrugged" and compare the situation to modern times. Ayn Rand, GRHS, was quite prophetic when she wrote that back in the 1950s.

[laserlight]

What I was looking for is the speed of SQL data bases adding Rows and the space SQL uses to store rows of data.

The results will vary between specific relational database systems, their configurations, hardware and OS used, the specifics of the test, etc. If you do a search online to look for benchmark comparisons between various systems, you'll find endless critique on the benchmarks themselves, even when those doing the benchmarks try to be as objective as they can.

I changed the number of records to be added to 50,000 and ADAM took nearly a second to do it. Still twice as fast as you were able to achieve with adding rows to an SQL data base.

For all we know if you do the test on your hardware, it will be twice as fast as ADAM... or maybe it will take several seconds. For example, using the same modified shell script, I get a wall clock time of under 1.6s for 50000 rows, i.e., twice as many rows was not twice as slow.

It used less than 26 Megabytes of disk space. How much space does an SQL data base with 512 byte rows take up on the hard drive?

Again, the results will vary between specific relational database systems, their configurations, hardware and OS used, the specifics of the test, etc. The notion of "512 byte rows" also does not quite apply, e.g., unless otherwise specified, SQLite3 has a rowid column for each table. This column would be indexed, possibly requiring additional space beyond simple storage of an integer, but it means extremely fast row retrieval by rowid. Then there is likely to be other meta-data concerning tables, columns and specific fields, along with the various structures like B-tree that constitute the database itself, things that cater to much more than just insertions into the database. In my case, I get a 55 MB database file, but a little experimenting shows that the storage used is not linear, e.g., changing the first argument to the shell script to 250 reduces the size to just 50 MB, but changing it to 240 reduces it to 25 MB. Hence, it looks easy to compare: 26 MB versus 55 MB (and maybe versus 24.4 MB as a baseline for 50000 * 512 B), but in reality it would be naive since databases are used for much more than just insertion of records consisting of 512 byte strings.

We've wandered all over the place here so lets get back to the reason I started this thread. What I was really looking for is the expertise to help me create a server that allows clients to create and access an ADAM file. Since no-one seems to be interested in real software development I probably won't be posting much anymore. What I've done so far seems to be a waste of time.

Agreed. It is unlikely that those around here who do have the particular expertise to help you will be inclined to do so given that you don't seem to have convinced anyone here that what you claim of ADAM is indeed accurate and worth the effort. I suggest that instead of dismissing what is current, you actually take time to experiment with them (and not just insertions, but the whole hog, e.g., retrieval, updating and deletions, and actually making use of foreign keys). Do not just look at relational database systems in current use, but also try out database systems with other approaches (particularly key-value stores).

[Will1]

The results will vary between specific relational database systems, their configurations, hardware and OS used, the specifics of the test, etc. If you do a search online to look for benchmark comparisons between various systems, you'll find endless critique on the benchmarks themselves, even when those doing the benchmarks try to be as objective as they can.

For all we know if you do the test on your hardware, it will be twice as fast as ADAM... or maybe it will take several seconds. For example, using the same modified shell script, I get a wall clock time of under 1.6s for 50000 rows, i.e., twice as many rows was not twice as slow.

My tests were done on a Pentium 4 processor, on a Dell GX 270, not the fastest PC around. Do you think that I could use "MYSQL" and build a server using ADAM as an engine for accessing data bases?

Again, the results will vary between specific relational database systems, their configurations, hardware and OS used, the specifics of the test, etc. The notion of "512 byte rows" also does not quite apply, e.g., unless otherwise specified, SQLite3 has a rowid column for each table. This column would be indexed, possibly requiring additional space beyond simple storage of an integer, but it means extremely fast row retrieval by rowid. Then there is likely to be other meta-data concerning tables, columns and specific fields, along with the various structures like B-tree that constitute the database itself, things that cater to much more than just insertions into the database. In my case, I get a 55 MB database file, but a little experimenting shows that the storage used is not linear, e.g., changing the first argument to the shell script to 250 reduces the size to just 50 MB, but changing it to 240 reduces it to 25 MB. Hence, it looks easy to compare: 26 MB versus 55 MB (and maybe versus 24.4 MB as a baseline for 50000 * 512 B), but in reality it would be naive since databases are used for much more than just insertion of records consisting of 512 byte strings.

Agreed, but it was just a test to see if the conversion from a DOS program to a Windows 32 bit program works and it and my other tests are working fine. (Note that I had to get around the "memory protection" feature of Windows and I did it with some success.)

Agreed. It is unlikely that those around here who do have the particular expertise to help you will be inclined to do so given that you don't seem to have convinced anyone here that what you claim of ADAM is indeed accurate and worth the effort. I suggest that instead of dismissing what is current, you actually take time to experiment with them (and not just insertions, but the whole hog, e.g., retrieval, updating and deletions, and actually making use of foreign keys). Do not just look at relational database systems in current use, but also try out database systems with other approaches (particularly key-value stores).

Agreed again: I haven't tested MASM32 ADAM with RRDS files but it will probably work. I have tested it extensively with KSDS
which are much more useful than RRDS files anyway. The "insertions" and other test that I have done were with KSDS files.

[stahta01]

Key Sequenced Data Set (KSDS)

Key Sequenced Data Set - Wikipedia, the free encyclopedia

[Will1]

Key Sequenced Data Set (KSDS)

Key Sequenced Data Set - Wikipedia, the free encyclopedia

OK, I'll give you a history of how I may have been the original designer of VSAM.

In 1969 I was working as a Customer Engineer for IBM in Denver. One day in 1969 I was told to go check on a problem that the !st National Bank of Denver had on their 360 system. The CE who normally serviced that account was out of town. I went and found out that the problem happened every now and then but I was able to track it down and find the hardware that caused the problem and replace it. The CIO wanted to talk to me. They were using the IBM Indexed Sequential Accass Method and it was notoriously a dog. He wanted to know if I could design a "Direct Access Method" to replace it.

I told him I thought so and he hired me away from IBM. I started work on the Acces Method. A few weeks later I got called into his office to discuss the project. There were 2 IBM Sysems Engineers there so I asked why and was told that they would review what I was doing and let them know if it was feasable. I said OK because I wouldn't own the Access Method, the bank would.

I laid out what was a very similar Access Method to VSAM except that I used "tracks" instad of "control intervals" (Whatever they are). Within a week the bank CIO layed me off and I figured that the IMB SEs told him it wouldn't work.

In the early 1970s I was working on a deal I made with a company that hada a 360 system to develop the father of ADAM was working on that when IBM announced the release of VSAM. Kind of a co-incidence don't you think.

KSDS is Key Sequenced Data sets, ESDS is Entry Sequrnced Data Sets, and RRDS is Relative Record Data Sets.

[phantomotap]

For example, using the same modified shell script, I get a wall clock time of under 1.6s for 50000 rows, i.e., twice as many rows was not twice as slow.

O_o

There is a fairly significant bug in my script. I have no excuse; I simply forgot to account for the fact that a hexadecimal representation uses two characters per byte.

You can try this version if you want to pretend, for whatever reason, to compare records of a more similar size.

./sql.sh 512 100000
timing build

real 0m0.669s
user 0m0.567s
sys  0m0.100s
99M  /tmp/tmp.TsuoZOn4uW/example.db
keys 100000

timing query 0

real 0m0.051s
user 0m0.017s
sys  0m0.033s

[Edit]
I added "begin"/"commit" lines to the build file.
[/Edit]

Soma

#!/bin/bash
#how many keys/searches are we going to fake
QUERIES=$3
#lazy attempt to account for the overhead of $rowid and $key
ADJUST=$4

if [[ -z "$ADJUST" ]]; then
  ADJUST=6
fi

if [[ -z "$QUERIES" ]]; then
  QUERIES=1
fi

if [[ "$1" -le "$ADJUST" ]]; then
  echo "\$1 needs to be greater than $ADJUST"
  exit 1
fi

COUNT=(0)
SCRATCH=$(mktemp -d)
trap 'rm -rf $SCRATCH' EXIT

build() {
  local KEY
  local VALUE
  echo "CREATE TABLE mytable(key INT, value VARCHAR);"
  echo "BEGIN;"
  for KEY in $(seq $2); do
    KEY="$(dd if=/dev/urandom bs=1 count=1 2>/dev/null | hexdump -e \"%i\" 2>/dev/null)"
    KEY=$((KEY % $QUERIES))
    COUNT[$KEY]=$((COUNT[$KEY] + 1))
    VALUE=$1
    VALUE=$((VALUE + 1))
    VALUE="$(dd if=/dev/urandom bs=1 count=$((VALUE / 2)) 2>/dev/null | hexdump -e \"%.2x\" 2>/dev/null | colrm $((VALUE - ADJUST)))"
    echo "INSERT INTO mytable VALUES($KEY, \"$VALUE\");"
  done
  echo "COMMIT;"
}

query() {
  echo "timing query $1"
  echo "SELECT value FROM mytable WHERE (key==$1);" | sqlite3 "$SCRATCH/example.db" 1>/dev/null 2>/dev/null
}

echo "warning generating the build instructions may take some time"
echo "sampling /dev/urandom like this is pretty dumb"

build $1 $2 1>$SCRATCH/build

echo "timing build"
time sqlite3 "$SCRATCH/example.db" <$SCRATCH/build
du -h "$SCRATCH/example.db"
echo -e "keys\t${COUNT[@]}\n"

for CURSOR in $(seq 0 $((QUERIES - 1))); do
  time query $CURSOR
  echo
done