Thread: Human-readable data storage format - tips?

  1. #1
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654

    Human-readable data storage format - tips?

    I am trying to separate data from code in order to reduce bugs, development time and sanity. So basically what I want is a file that describes an event system. It describes WHAT happens when WHAT happens, basically. Now, I need a file format to store this so I can parse it and apply it later in the code.

    To that end, I am searching for good data storage formats. The requirements that I have are:

    - Human readable (I am going to write this, obviously)
    - Validation support (e.g. schemas)
    - Good FREE tool support for validating the files (I am NOT going to receive this data dynamically, so I DON'T need a library)
    - Easy to use
    - Must be support for parsing and reading from VBA (for use inside Access)
    - Would be nice if there is a good IDE/Editor for writing the file format

    So I've been looking some, and the formats I know are: XML, JSON and YAML.

    XML is big, fast and difficult to use.

    I can't seem to find any good tools for JSON. All I find are libraries, which are mostly useless to me. I am not planning on receiving data dynamically, so I don't want to write code to verify it. I just want a good tool.

    YAML gives almost zero results when looking for tools.

    I may relent on the library bit if it's easy to implement and its output rich (I don't need something that just says that the file doesn't pass validation), but I'd rather not code it.

    So that's why I thought I'd borrow some experience of some of the members here to validate my options. Keep in mind, this isn't a commercial project, so I don't intend to pay for simple tools that works with very simple file formats.

    I probably don't need to say this, but I'll mention it anyway. The tools must work under Windows. Not Linux. Not Mac OS. Web is fine.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  2. #2
    Master Apprentice phantomotap's Avatar
    Join Date
    Jan 2008
    Posts
    5,108
    O_o

    XML): I wouldn't recommend it. You'll not find it at all difficult with the right library/tools, but if you ever do need to edit by hand, well, the others are more palatable without tools. You would have access to extremely popular tools, and native expression for schema in "XSD".

    YAML): I don't like YAML, but I'd prefer it over XML. However, you are wanting a way to verify the data at a higher level, and YAML has no real native support for that.

    JSON): I use JSON for basically everything that doesn't need to be binary data. "JSON" also has a pretty widely accepted form of "schema" with some tool integration. Much like the "XSD", the "JSON" schema I work with is itself expressed in "JSON" which itself has a standard schema expressed in "JSON". (In other words, if you decide to write your own code, the schema definition language, the validator for definitions, and the "JSON" parser are pass through the same code.) I question that you were unable to find tools; seriously, take another pass at your search engine with "html5", "JSON", and "viewer".

    Soma
    “Salem Was Wrong!” -- Pedant Necromancer
    “Four isn't random!” -- Gibbering Mouther

  3. #3
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    I find lots of tools to view, uglify, parse, etc json, but none to actually verify some json against a schema. All that I can find are libraries for different languages (obviously VBA isn't included, either).
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  4. #4
    Master Apprentice phantomotap's Avatar
    Join Date
    Jan 2008
    Posts
    5,108
    O_o

    JSON Schema - Simple Example
    JSON Schema Software

    You should try looking for C instead of C++.

    Soma
    “Salem Was Wrong!” -- Pedant Necromancer
    “Four isn't random!” -- Gibbering Mouther

  5. #5
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    You should know I have no love for C... unfortunately JSON is treated much like a "dynamic web language", causing there not to be many useful tools available for it. I have already seen that page and discarded it, but I will keep it around for future reference.
    I am going to look into doing it in XML instead because I believe the tools are there (Visual Studio can easily generate a basic schema from an XML file; good luck doing that for JSON), and it's easy to parse in VBA.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  6. #6
    Registered User MutantJohn's Avatar
    Join Date
    Feb 2013
    Posts
    2,665
    Why do you have no love for C? How can you say such a thing? That's blasphemous!

  7. #7
    Master Apprentice phantomotap's Avatar
    Join Date
    Jan 2008
    Posts
    5,108
    I am going to look into doing it in XML instead because I believe the tools are there (Visual Studio can easily generate a basic schema from an XML file; good luck doing that for JSON), and it's easy to parse in VBA.
    O_o

    No. You just aren't looking.

    *shrug*

    Which is fine; you should really try all three anyway. Even if you really like XML or JSON, you may find the simplicity of YAML more palatable, but if you don't look around more than you have, you aren't going to find anything.

    I found a free tool to infer a JSON schema from JSON data using the standard I've referenced in the first link from my first search of the internet.

    Soma
    “Salem Was Wrong!” -- Pedant Necromancer
    “Four isn't random!” -- Gibbering Mouther

  8. #8
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Quote Originally Posted by phantomotap View Post
    No. You just aren't looking.
    I am looking! But maybe I suck at looking apparently...
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Txt file and data storage
    By radnik in forum C++ Programming
    Replies: 52
    Last Post: 06-24-2008, 09:16 AM
  2. C++ data storage
    By aewing30 in forum C++ Programming
    Replies: 5
    Last Post: 04-28-2008, 02:19 PM
  3. ListBox Extra Data Storage
    By Welder in forum Windows Programming
    Replies: 1
    Last Post: 11-01-2007, 01:46 PM
  4. Data Storage Method
    By JupiterV2 in forum C Programming
    Replies: 1
    Last Post: 04-27-2007, 04:15 PM
  5. data storage
    By azzalicious in forum C++ Programming
    Replies: 4
    Last Post: 10-15-2001, 06:33 PM