jay
/
binary-data-format
mirror of https://github.com/jsrobson10/binary-data-format
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Updated the README, added support for non-real integers, fixed issues 

when trying to parse garbage data, cleaned up some code.
This commit is contained in:
jsrobson10 2020-09-09 14:02:38 +10:00
parent 04657bc6ad
commit 960d0c3b17
14 changed files with 568 additions and 482 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

238
README.md
View File

@ -8,23 +8,19 @@
- <a href="#creating-an-object">Creating an object</a>
- <a href="#arrays">Arrays</a>
- <a href="#named-lists">Named lists</a>
- <a href="#saving-classes">Saving classes</a>
- <a href="#implementation-details">Implementation details</a>
- <a href="#human-readable-representation">Human readable representation</a>
- <a href="#special-notes">Special notes</a>
### Overview
Binary Data Format (or BDF) is designed to store data in a tree-like binary structure,
like Notch's NBT format, but also open source and free like JSON. The format is
fast and allows multiple data types. It uses 32-bit integers, so BDF files can
be fast and work well on 32-bit systems, but have a maximum size of 2 GB.
BDF allows human readable serialization to see what is going on for debugging
purposes, but it currently can't parse the human readable serialized string to an object.
Binary Data Format (BDF) is a statically typed data representation
format. It was made to be free, fast, compact, and seamlessly
convertable between its human readable and binary representations.
### Languages
- Java
- <a href="https://github.com/jsrobson10/BdfCpp">C++</a>
- Java
### Data types
@ -70,8 +66,15 @@ int v = bdf.getInteger();
// Set an integer
bdf.setInteger(5);
// Set a "smart" integer
bdf.setSmartInteger(53);
// Set an integer with an automatic type
bdf.setAutoInt(53);
// Set a primitive array of ints
int intArray[] = {3, 4, 5, 6};
bdf.setIntegerArray(intArray);
// Get a byte array
byte[] byteArray = bdf.getByteArray();
// Get the type of variable of the object
int type = bdf.getType();
@ -83,38 +86,27 @@ if(type == BdfTypes.INTEGER)
}
// Serialize the BDF object
IBdfDatabase data = bdf.serialize();
byte[] data = bdf->serialize(&data, &data_size);
// Load another BDF object with the serialized bytes
BdfObject bdf2 = new BdfObject(new BdfDatabase(data));
// Load another reader object from the serialized bytes
BdfReader reader2 = new BdfReader(data);
```
/*
A reader object can be serialized to the human readable
representation as a string or sent over a stream
*/
reader2.serializeHumanReadable(System.out, new BdfIndent("\t", "\n"));
String data_hr = reader2.serializeHumanReadable(new BdfIndent("\t", "\n"));
A file manager instance can be used in the same way as a reader object,
but it also needs a String parameter for the path of the file. The file
manager instance also has the capacity to use compression (by default this
uses the GZIP compression algorithm).
```java
// Open a file with compression enabled
BdfFileManager reader = new BdfFileManager("file.bdf", true);
// Save the database
reader.saveDatabase();
// The file can be casted to a BdfReader
BdfReader reader2 = (BdfReader) reader;
// Can be used just as any reader instance
BdfObject bdf = reader.getObject();
// A reader object can be loaded from a human readable object
BdfReader reader3 = new BdfReaderHuman(data_hr);
```
### Arrays
Arrays can be used to store lists of information, they hold instances of
BdfObject. Arrays have support for Iterators and are an instance of Iterable.
Arrays can be used to store chunks of information, they hold instances of
BdfObject. Arrays can also be iterated over just like any other array.
```java
@ -134,17 +126,11 @@ array.remove(3);
array.set(4, bdf.newObject().setString("A String"));
// Add an object to an array
array.add(bdf.newObject().setByte(53));
array.add(bdf.newObject().setByte((byte)53));
// Set the array to the bdf object
bdf.setArray(array);
// Iterate over an array
for(BdfObject o : array)
{
}
```
### Named lists
@ -156,26 +142,26 @@ can be created similar to an array.
```java
BdfReader reader = new BdfReader();
BdfObject bdf = new BdfObject();
BdfObject bdf = reader.getObject();
// New named list
BdfNamedList nl = bdf.newNamedList();
BdfNamedList list = bdf.newNamedList();
// Set an element to the named list
nl.set("key1", bdf.newObject().setInteger(5));
list.set("key1", bdf.newObject().setInteger(5));
// Use ids instead of strings for optimisation
// if set/get is being called multiple times
// on the same key.
int key2 = nl.getKeyLocation("key2");
nl.set(key2, bdf.newObject().setFloat(42.0F));
int key2 = bdf.getKeyLocation("key2");
list.set(key2, bdf.newObject().setFloat(42.0F));
// Get an elements value
int v = list.get("key1").getInteger();
// Check if an element exists
boolean has_key = list.contains("key1");
bool has_key = list.contains("key1");
// Get the lists keys
int[] keys = list.getKeys();
@ -184,64 +170,132 @@ int[] keys = list.getKeys();
for(int key : keys)
{
// Get the keys name
String key_name = nl.getKeyName(key);
String key_name = bdf.getKeyName(key);
}
```
### Further optimisations
### Human readable representation
A big part of binary data format is the human readable
representation. It has a JSON-like syntax.
This can be used with config files and to modify/view
binaries. A big advantage to using the human readable
representation in configuration files is its support
for comments.
### Implementation details
```hbdf
All integer data types are in the Big Endian layout.
/*
A Named List is represented
by an opening tag and a closing
tag { }
*/
{
/*
A key value pair can be stored
within a Named List with a string
property
*/
"hello": "world",
**Flags (1 unsigned byte)**
This holds 3 values:
- Type (0-17)
- Size type (0-2)
- Parent payload (0-2)
/*
Integers can be stored here too.
They have a character at the end
to say what type they are.
**Type**
```
0: UNDEFINED (0 bytes)
The tag at the end can be:
- I: Integer - a value between -2^31 and 2^31 - 1
- S: Short - a value between -32768 and 32767
- L: Long - a value between -2^63 and 2^63 - 1
- B: Byte - a value between -128 and 127
- D: Double - has 15 decimal digits of precision
- F: Float - has 7 decimal digits of precision
*/
"number": 42I,
"byte": -23B,
"decimal": 73.5D,
1: BOOLEAN (1 byte, 0x00 or 0x01)
2: INTEGER (4 bytes)
3: LONG (8 bytes)
4: SHORT (2 bytes)
5: BYTE (1 byte)
6: DOUBLE (8 bytes)
7: FLOAT (4 bytes)
/*
This is a boolean. It can
be true or false.
*/
"boolTest": false,
8: STRING
9: ARRAY
10: NAMED_LIST
/*
Primitive arrays are represented
by a type, an opening tag, and a
closing tag. They are like an array
but they contain only 1 data type.
11: ARRAY_BOOLEAN
12: ARRAY_INTEGER
13: ARRAY_LONG
14: ARRAY_SHORT
15: ARRAY_BYTE
16: ARRAY_DOUBLE
17: ARRAY_FLOAT
The tag at the start can be:
- int
- short
- long
- byte
- double
- float
- bool
*/
"intArray": int (
64I, 42I, 63I,
22I, 96I, -12I,
),
/*
The double and float types support
Infinity, -Infinity, and NaN.
They also support both really
high and really low value numbers.
*/
"doubleArray": double (
42.5D, -20D, 400D,
NaND, -InfinityD, InfinityD,
5.3e-200F, 4e+500F, 2.2e200F,
)
/*
Arrays are enclosed by an opening
tag and a closing tag [ ]
Like the Named List, it can hold
any data type.
*/
"people": [
{"name": "foo", "age": 60B},
{"name": "bar", "age": 21B},
],
// This is a single-line comment
/* This is a multi-line comment */
}
```
**Size Type**
This value holds info for how big the size of
the size of the payload is, in bytes. The purpose
of this is to reduce the size as much as possible
by throwing out unneccicary zeros.
### Special notes
**Object**
- Flags (unsigned byte, 1 byte)
- Size (variable length)
- Payload (Any type, variable length)
Don't mix bdf types between different
readers, this will cause problems.
**NamedList**
- Key ID (variable length)
- Payload (Object, variable length)
```java
BdfReader reader1 = new BdfReader();
BdfReader reader2 = new BdfReader();
BdfObject bdf1 = reader1.getObject();
BdfObject bdf2 = reader2.getObject();
// Don't do this
bdf1.setNamedList(bdf2.newNamedList());
// Or this
bdf1.setArray(bdf2.newArray());
// Or this
BdfNamedList nl = bdf1.newArray();
nl.set("illegal", bdf2.newObject().setString("action"));
```
**Array**
- Payload (Object, variable length)

9
src/bdf/classes/IBdfClassManager.java
View File

@ -1,9 +0,0 @@
package bdf.classes;
import bdf.types.BdfObject;
public interface IBdfClassManager
{
public void BdfClassLoad(BdfObject bdf);
public void BdfClassSave(BdfObject bdf);
}

89
src/bdf/data/BdfStringPointer.java
View File

@ -7,6 +7,11 @@ public class BdfStringPointer
char[] data;
int offset;
public BdfStringPointer(char[] data, int offset) {
this.data = data;
this.offset = offset;
}
public BdfStringPointer getPointer(int offset) {
return new BdfStringPointer(data, this.offset + offset);
}
@ -46,11 +51,6 @@ public class BdfStringPointer
return getCharArray(0, length);
}
public BdfStringPointer(char[] data, int offset) {
this.data = data;
this.offset = offset;
}
public char getChar(int i) {
return data[offset + i];
}
@ -61,7 +61,7 @@ public class BdfStringPointer
public void ignoreBlanks()
{
while(true)
for(;;)
{
if(offset >= data.length) {
throw BdfError.createError(BdfError.ERROR_END_OF_FILE, this);
@ -69,7 +69,54 @@ public class BdfStringPointer
char c = getChar();
if(!(c == '\n' || c == '\t' || c == ' ')) {
// Comments
if(c == '/' && offset < data.length)
{
char c2 = getChar(1);
// Line comment
if(c2 == '/')
{
for(;;)
{
if(offset + 1 >= data.length) {
break;
}
increment();
c = getChar();
if(c == '\n') {
break;
}
}
}
// Multi-line comment
else if(c2 == '*')
{
for(;;)
{
if(offset + 1 >= data.length) {
throw BdfError.createError(BdfError.ERROR_UNESCAPED_COMMENT, getPointer(-1));
}
increment();
c = getChar();
if(c == '*' && offset < data.length && getChar(1) == '/') {
increment();
break;
}
}
}
else {
return;
}
}
else if(!(c == '\n' || c == '\t' || c == ' ')) {
return;
}
@ -87,10 +134,10 @@ public class BdfStringPointer
increment();
String str = "";
while(true)
for(;;)
{
if(offset >= data.length) {
throw BdfError.createError(BdfError.ERROR_END_OF_FILE, this);
throw BdfError.createError(BdfError.ERROR_UNESCAPED_STRING, this);
}
char c = getChar();
@ -98,27 +145,35 @@ public class BdfStringPointer
// Check for back slashes
if(c == '\\')
{
increment(1);
increment();
c = getChar();
switch(c)
{
case 'n':
str += "\n";
increment();
break;
case 't':
str += "\t";
increment();
break;
case '"':
str += "\"";
increment();
break;
case '\\':
str += "\\";
increment();
break;
case '\n':
str += "\n";
increment();
break;
case 'u': // \u0000
{
if(offset + 5 >= data.length) {
throw BdfError.createError(BdfError.ERROR_END_OF_FILE, getPointer(1));
throw BdfError.createError(BdfError.ERROR_UNESCAPED_STRING, getPointer(1));
}
char[] hex = getCharArray(1, 4);
@ -154,6 +209,10 @@ public class BdfStringPointer
}
}
else if(c == '\n') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, this);
}
else if(c == '"') {
increment();
break;
@ -170,16 +229,12 @@ public class BdfStringPointer
public boolean isNext(String check)
{
if(check.length() + offset >= data.length) {
if(check.length() + offset > data.length) {
return false;
}
for(int i=0;i<check.length();i++)
{
if(offset + i >= data.length) {
throw BdfError.createError(BdfError.ERROR_END_OF_FILE, this);
}
char c = getChar(i);
c = (char)((c >= 'A' && c <= 'Z') ? (c + 32) : c);
@ -219,6 +274,8 @@ public class BdfStringPointer
continue;
case '.':
continue;
case '-':
continue;
}
if(c >= '0' && c <= '9') {

1
src/bdf/file/.gitignore vendored
View File

@ -1 +0,0 @@
/BdfFileManager.class

89
src/bdf/file/BdfFileManager.java
View File

@ -1,89 +0,0 @@
package bdf.file;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.zip.DeflaterOutputStream;
import java.util.zip.GZIPOutputStream;
import bdf.data.BdfDatabase;
import bdf.types.BdfObject;
import bdf.types.BdfReader;
import bdf.util.FileHelpers;
public class BdfFileManager extends BdfReader
{
protected String path;
private boolean compressed;
private static BdfDatabase init(String path, boolean compressed)
{
// Get the file
File file = new File(path);
// Does the file have read access
if(file.canRead())
{
if(compressed)
{
// Return the files contents as a database
return new BdfDatabase(FileHelpers.readAllCompressedIgnoreErrors(path));
}
else
{
// Return the files contents as a database
return new BdfDatabase(FileHelpers.readAllIgnoreErrors(path));
}
}
// Return an empty database if there is no read access
return new BdfDatabase(0);
}
public BdfFileManager(String path, boolean compressed) {
super(init(path, compressed));
this.compressed = compressed;
this.path = path;
}
public BdfFileManager(String path) {
this(path, false);
}
public void saveDatabase(String path)
{
try
{
// Get the file handler
File file = new File(path);
// Create the parent directories
file.getAbsoluteFile().getParentFile().mkdirs();
// Get the database file for output
OutputStream out = new FileOutputStream(path);
if(compressed) {
out = new GZIPOutputStream(out);
}
// Write the database to the file
BdfDatabase db = this.serialize();
db.writeToStream(out);
// Close the file output stream
out.close();
}
catch(IOException e) {
return;
}
}
public void saveDatabase() {
this.saveDatabase(this.path);
}
}

9
src/bdf/types/BdfArray.java
View File

@ -8,16 +8,13 @@ import java.util.Iterator;
import bdf.data.BdfStringPointer;
import bdf.data.IBdfDatabase;
import bdf.util.BdfError;
import bdf.util.DataHelpers;
public class BdfArray implements IBdfType, Iterable<BdfObject>
{
protected ArrayList<BdfObject> elements;
protected BdfLookupTable lookupTable;
BdfArray(BdfLookupTable lookupTable, BdfStringPointer ptr)
{
this.lookupTable = lookupTable;
this.elements = new ArrayList<BdfObject>();
ptr.increment();
@ -55,7 +52,6 @@ public class BdfArray implements IBdfType, Iterable<BdfObject>
BdfArray(BdfLookupTable lookupTable, int size)
{
this.lookupTable = lookupTable;
this.elements = new ArrayList<BdfObject>(size);
for(int i=0;i<size;i++) {
@ -65,7 +61,6 @@ public class BdfArray implements IBdfType, Iterable<BdfObject>
BdfArray(BdfLookupTable lookupTable, IBdfDatabase data)
{
this.lookupTable = lookupTable;
this.elements = new ArrayList<BdfObject>();
// Create an iterator value to loop over the data
@ -77,6 +72,10 @@ public class BdfArray implements IBdfType, Iterable<BdfObject>
// Get the size of the object
int size = BdfObject.getSize(data.getPointer(i));
if(size <= 0 || i + size > data.size()) {
return;
}
// Get the object
BdfObject object = new BdfObject(lookupTable, data.getPointer(i, size));

12
src/bdf/types/BdfLookupTable.java
View File

@ -1,11 +1,7 @@
package bdf.types;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Comparator;
import bdf.data.IBdfDatabase;
import bdf.util.DataHelpers;
@ -31,6 +27,10 @@ class BdfLookupTable implements IBdfType
int key_size = 0xff & database.getByte(i);
i += 1;
if(i + key_size > database.size()) {
return;
}
keys.add(database.getBytes(i, key_size));
i += key_size;
@ -50,6 +50,10 @@ class BdfLookupTable implements IBdfType
return keys.size() - 1;
}
boolean hasLocation(int location) {
return location >= 0 && location < keys.size();
}
byte[] getName(int location) {
return keys.get(location);
}

10
src/bdf/types/BdfNamedList.java
View File

@ -3,7 +3,6 @@ package bdf.types;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import bdf.data.BdfStringPointer;
@ -94,6 +93,11 @@ public class BdfNamedList implements IBdfType
IBdfDatabase flag_ptr = data.getPointer(i);
int object_size = BdfObject.getSize(flag_ptr);
byte key_size_bytes = BdfObject.getParentFlags(flag_ptr);
if(object_size <= 0 || i + object_size >= data.size()) {
return;
}
BdfObject object = new BdfObject(lookupTable, data.getPointer(i, object_size));
i += object_size;
@ -127,6 +131,10 @@ public class BdfNamedList implements IBdfType
break;
}
if(!lookupTable.hasLocation(key)) {
return;
}
i += key_size;
// Create a new element and save some data to it

231
src/bdf/types/BdfObject.java
View File

@ -1,6 +1,5 @@
package bdf.types;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.ByteBuffer;
@ -10,7 +9,6 @@ import bdf.data.BdfStringPointer;
import bdf.data.IBdfDatabase;
import bdf.util.BdfError;
import bdf.util.DataHelpers;
import tests.Tests;
public class BdfObject implements IBdfType
{
@ -42,10 +40,25 @@ public class BdfObject implements IBdfType
database = database.getPointer(size_bytes);
}
if(database.size() < 0) {
database = new BdfDatabase(0);
type = BdfTypes.UNDEFINED;
return;
}
// Set the object variable if there is an object specified
if(type == BdfTypes.STRING) object = database.getString();
if(type == BdfTypes.ARRAY) object = new BdfArray(lookupTable, database);
if(type == BdfTypes.NAMED_LIST) object = new BdfNamedList(lookupTable, database);
switch(type)
{
case BdfTypes.STRING:
object = database.getString();
break;
case BdfTypes.ARRAY:
object = new BdfArray(lookupTable, database);
break;
case BdfTypes.NAMED_LIST:
object = new BdfNamedList(lookupTable, database);
break;
}
if(object != null) {
database = null;
@ -73,6 +86,7 @@ public class BdfObject implements IBdfType
return;
}
boolean isDecimalArray = false;
boolean isPrimitiveArray = false;
byte type = 0;
@ -104,11 +118,13 @@ public class BdfObject implements IBdfType
else if(ptr.isNext("double")) {
type = BdfTypes.ARRAY_DOUBLE;
isPrimitiveArray = true;
isDecimalArray = true;
}
else if(ptr.isNext("float")) {
type = BdfTypes.ARRAY_FLOAT;
isPrimitiveArray = true;
isDecimalArray = true;
}
// Deserialize a primitive array
@ -131,7 +147,17 @@ public class BdfObject implements IBdfType
for(;;)
{
if(ptr2.isNext("true") || ptr2.isNext("false")) {
if(ptr2.getChar() == ')') {
ptr2.increment();
break;
}
if(
ptr2.isNext("true") || ptr2.isNext("false") ||
ptr2.isNext("infinityd") || ptr2.isNext("-infinityd") ||
ptr2.isNext("infinityf") || ptr2.isNext("-infinityf") ||
ptr2.isNext("nand") || ptr2.isNext("nanf"))
{
size += 1;
}
@ -139,9 +165,17 @@ public class BdfObject implements IBdfType
{
for(;;)
{
if(ptr2.getDataLocation() >= ptr2.getDataLength()) {
throw BdfError.createError(BdfError.ERROR_END_OF_FILE, ptr2.getPointer(-1));
}
c = ptr2.getChar();
if(c >= '0' && c <= '9' || c == '.' || c == 'e' || c == 'E' || c == '-') {
if(c >= 'a' && c <= 'z') {
c -= 32;
}
if(c >= '0' && c <= '9' || ((c == '.' || c == 'E') && isDecimalArray) || c == '+' || c == '-') {
ptr2.increment();
continue;
}
@ -198,6 +232,11 @@ public class BdfObject implements IBdfType
for(int i=0;;i++)
{
if(ptr.getChar() == ')') {
ptr.increment();
break;
}
if(ptr.isNext("true"))
{
if(type != BdfTypes.ARRAY_BOOLEAN) {
@ -218,6 +257,66 @@ public class BdfObject implements IBdfType
a[i] = false;
}
else if(ptr.isNext("infinityd"))
{
if(type != BdfTypes.ARRAY_DOUBLE) {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
double[] a = (double[]) array;
a[i] = Double.POSITIVE_INFINITY;
}
else if(ptr.isNext("-infinityd"))
{
if(type != BdfTypes.ARRAY_DOUBLE) {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
double[] a = (double[]) array;
a[i] = Double.NEGATIVE_INFINITY;
}
else if(ptr.isNext("nand"))
{
if(type != BdfTypes.ARRAY_DOUBLE) {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
double[] a = (double[]) array;
a[i] = Double.NaN;
}
else if(ptr.isNext("infinityf"))
{
if(type != BdfTypes.ARRAY_FLOAT) {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
float[] a = (float[]) array;
a[i] = Float.POSITIVE_INFINITY;
}
else if(ptr.isNext("-infinityf"))
{
if(type != BdfTypes.ARRAY_FLOAT) {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
float[] a = (float[]) array;
a[i] = Float.NEGATIVE_INFINITY;
}
else if(ptr.isNext("nanf"))
{
if(type != BdfTypes.ARRAY_FLOAT) {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
float[] a = (float[]) array;
a[i] = Float.NaN;
}
else
{
// Parse a number
@ -227,11 +326,15 @@ public class BdfObject implements IBdfType
{
c = ptr.getChar();
if(c >= 'a' && c <= 'z') {
c -= 32;
}
if(ptr.getDataLocation() > ptr.getDataLength()) {
throw BdfError.createError(BdfError.ERROR_END_OF_FILE, ptr);
}
if(c >= '0' && c <= '9' || c == '.' || c == 'e' || c == 'E' || c == '-') {
if(c >= '0' && c <= '9' || ((c == '.' || c == 'E') && isDecimalArray) || c == '+' || c == '-') {
ptr.increment();
number += c;
continue;
@ -372,50 +475,103 @@ public class BdfObject implements IBdfType
return;
}
if(ptr.isNext("infinityd")) {
setDouble(Double.POSITIVE_INFINITY);
return;
}
if(ptr.isNext("-infinityd")) {
setDouble(Double.NEGATIVE_INFINITY);
return;
}
if(ptr.isNext("nand")) {
setDouble(Double.NaN);
return;
}
if(ptr.isNext("infinityf")) {
setFloat(Float.POSITIVE_INFINITY);
return;
}
if(ptr.isNext("-infinityf")) {
setFloat(Float.NEGATIVE_INFINITY);
return;
}
if(ptr.isNext("nanf")) {
setFloat(Float.NaN);
return;
}
if(ptr.isNext("undefined")) {
database = new BdfDatabase(0);
return;
}
// Parse a number
String number = "";
boolean isDecimal = false;
for(;;)
{
c = ptr.getChar();
ptr.increment();
if(c >= 'a' && c <= 'z') {
c -= 32;
}
if(ptr.getDataLocation() > ptr.getDataLength()) {
throw BdfError.createError(BdfError.ERROR_END_OF_FILE, ptr);
}
if(c >= '0' && c <= '9' || c == '.' || c == 'e' || c == 'E' || c == '-') {
if(c == '.' || c == 'E') {
isDecimal = true;
number += c;
continue;
}
switch(c)
{
case 'D':
setDouble(Double.parseDouble(number));
return;
case 'F':
setFloat(Float.parseFloat(number));
return;
case 'B':
setByte(Byte.parseByte(number));
return;
case 'S':
setShort(Short.parseShort(number));
return;
case 'I':
setInteger(Integer.parseInt(number));
return;
case 'L':
setLong(Long.parseLong(number));
return;
if(c >= '0' && c <= '9' || c == '+' || c == '-') {
number += c;
continue;
}
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
if(isDecimal && !(c == 'D' || c == 'F')) {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr.getPointer(-1));
}
try
{
switch(c)
{
case 'D':
setDouble(Double.parseDouble(number));
return;
case 'F':
setFloat(Float.parseFloat(number));
return;
case 'B':
setByte(Byte.parseByte(number));
return;
case 'S':
setShort(Short.parseShort(number));
return;
case 'I':
setInteger(Integer.parseInt(number));
return;
case 'L':
setLong(Long.parseLong(number));
return;
default:
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
}
catch(NumberFormatException e) {
throw BdfError.createError(BdfError.ERROR_NUMBER, ptr.getPointer(-number.length()));
}
}
}
@ -673,7 +829,7 @@ public class BdfObject implements IBdfType
break;
case BdfTypes.ARRAY_INTEGER: {
stream.write(("int(" + calcIndent(indent, it)).getBytes());
stream.write("int(".getBytes());
int[] array = this.getIntegerArray();
for(int i=0;i<array.length;i++) {
stream.write((indent.breaker + calcIndent(indent, it) + Integer.toString(array[i]) + "I").getBytes());
@ -684,7 +840,7 @@ public class BdfObject implements IBdfType
}
case BdfTypes.ARRAY_BOOLEAN: {
stream.write(("bool(" + calcIndent(indent, it)).getBytes());
stream.write("bool(".getBytes());
boolean[] array = this.getBooleanArray();
for(int i=0;i<array.length;i++) {
stream.write((indent.breaker + calcIndent(indent, it) + (array[i] ? "true" : "false")).getBytes());
@ -695,7 +851,7 @@ public class BdfObject implements IBdfType
}
case BdfTypes.ARRAY_SHORT: {
stream.write(("short(" + calcIndent(indent, it)).getBytes());
stream.write("short(".getBytes());
short[] array = this.getShortArray();
for(int i=0;i<array.length;i++) {
stream.write((indent.breaker + calcIndent(indent, it) + Short.toString(array[i]) + "S").getBytes());
@ -706,7 +862,7 @@ public class BdfObject implements IBdfType
}
case BdfTypes.ARRAY_LONG: {
stream.write(("long(" + calcIndent(indent, it)).getBytes());
stream.write("long(".getBytes());
long[] array = this.getLongArray();
for(int i=0;i<array.length;i++) {
stream.write((indent.breaker + calcIndent(indent, it) + Long.toString(array[i]) + "L").getBytes());
@ -717,7 +873,7 @@ public class BdfObject implements IBdfType
}
case BdfTypes.ARRAY_BYTE: {
stream.write(("byte(" + calcIndent(indent, it)).getBytes());
stream.write("byte(".getBytes());
byte[] array = this.getByteArray();
for(int i=0;i<array.length;i++) {
stream.write((indent.breaker + calcIndent(indent, it) + Byte.toString(array[i]) + "B").getBytes());
@ -728,7 +884,7 @@ public class BdfObject implements IBdfType
}
case BdfTypes.ARRAY_DOUBLE: {
stream.write(("double(" + calcIndent(indent, it)).getBytes());
stream.write("double(".getBytes());
double[] array = this.getDoubleArray();
for(int i=0;i<array.length;i++) {
stream.write((indent.breaker + calcIndent(indent, it) + Double.toString(array[i]) + "D").getBytes());
@ -739,7 +895,7 @@ public class BdfObject implements IBdfType
}
case BdfTypes.ARRAY_FLOAT: {
stream.write(("float(" + calcIndent(indent, it)).getBytes());
stream.write("float(".getBytes());
float[] array = this.getFloatArray();
for(int i=0;i<array.length;i++) {
stream.write((indent.breaker + calcIndent(indent, it) + Float.toString(array[i]) + "F").getBytes());
@ -1063,7 +1219,6 @@ public class BdfObject implements IBdfType
public BdfObject setString(String value) {
this.type = BdfTypes.STRING;
this.database = new BdfDatabase(value.getBytes());
this.object = value;
return this;
}

125
src/bdf/types/BdfReader.java
View File

@ -2,12 +2,10 @@ package bdf.types;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.ByteBuffer;
import bdf.data.BdfDatabase;
import bdf.data.BdfStringPointer;
import bdf.data.IBdfDatabase;
import bdf.util.DataHelpers;
@ -31,62 +29,77 @@ public class BdfReader
public BdfReader(IBdfDatabase database)
{
if(database.size() == 0) {
try
{
if(database.size() == 0) {
initNew();
return;
}
int upto = 0;
IBdfDatabase flag_ptr = database.getPointer(upto);
byte lookupTable_size_tag = BdfObject.getParentFlags(flag_ptr);
byte lookupTable_size_bytes = 0;
switch(lookupTable_size_tag)
{
case 0:
lookupTable_size_bytes = 4;
break;
case 1:
lookupTable_size_bytes = 2;
break;
case 2:
lookupTable_size_bytes = 1;
break;
}
if(lookupTable_size_bytes > database.size()) {
initNew();
return;
}
// Get the rest of the data
int bdf_size = BdfObject.getSize(flag_ptr);
if(bdf_size + lookupTable_size_bytes > database.size()) {
initNew();
return;
}
IBdfDatabase database_bdf = database.getPointer(upto, bdf_size);
upto += bdf_size;
// Get the lookup table
ByteBuffer lookupTable_size_buff = DataHelpers.getByteBuffer(database.getPointer(upto, lookupTable_size_bytes));
int lookupTable_size = 0;
switch(lookupTable_size_tag)
{
case 0:
lookupTable_size = lookupTable_size_buff.getInt();
break;
case 1:
lookupTable_size = 0xffff & lookupTable_size_buff.getShort();
break;
case 2:
lookupTable_size = 0xff & lookupTable_size_buff.get();
break;
}
if(lookupTable_size + lookupTable_size_bytes + bdf_size > database.size()) {
initNew();
return;
}
lookupTable = new BdfLookupTable(this, database.getPointer(lookupTable_size_bytes + upto, lookupTable_size));
bdf = new BdfObject(lookupTable, database_bdf);
}
catch(IndexOutOfBoundsException e) {
initNew();
return;
}
int upto = 0;
IBdfDatabase flag_ptr = database.getPointer(upto);
byte lookupTable_size_tag = BdfObject.getParentFlags(flag_ptr);
byte lookupTable_size_bytes = 0;
switch(lookupTable_size_tag)
{
case 0:
lookupTable_size_bytes = 4;
break;
case 1:
lookupTable_size_bytes = 2;
break;
case 2:
lookupTable_size_bytes = 1;
break;
}
// Get the rest of the data
int bdf_size = BdfObject.getSize(flag_ptr);
IBdfDatabase database_bdf = database.getPointer(upto, bdf_size);
upto += bdf_size;
// Get the lookup table
ByteBuffer lookupTable_size_buff = DataHelpers.getByteBuffer(database.getPointer(upto, lookupTable_size_bytes));
int lookupTable_size = 0;
switch(lookupTable_size_tag)
{
case 0:
lookupTable_size = lookupTable_size_buff.getInt();
break;
case 1:
lookupTable_size = 0xffff & lookupTable_size_buff.getShort();
break;
case 2:
lookupTable_size = 0xff & lookupTable_size_buff.get();
break;
}
lookupTable = new BdfLookupTable(this, database.getPointer(lookupTable_size_bytes + upto, lookupTable_size));
bdf = new BdfObject(lookupTable, database_bdf);
}
public static BdfReader readHumanReadable(String data)
{
BdfReader reader = new BdfReader();
reader.bdf = new BdfObject(reader.lookupTable, new BdfStringPointer(data.toCharArray(), 0));
return reader;
}
public BdfDatabase serialize()

29
src/bdf/types/BdfReaderHuman.java Normal file
View File

@ -0,0 +1,29 @@
package bdf.types;
import bdf.data.BdfStringPointer;
import bdf.util.BdfError;
public class BdfReaderHuman extends BdfReader
{
public BdfReaderHuman(String data)
{
BdfStringPointer ptr = new BdfStringPointer(data.toCharArray(), 0);
ptr.ignoreBlanks();
bdf = new BdfObject(lookupTable, ptr);
try {
ptr.ignoreBlanks();
}
catch(BdfError e) {
if(e.getType() == BdfError.ERROR_END_OF_FILE) {
return;
} else {
throw e;
}
}
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
}

28
src/bdf/util/BdfError.java
View File

@ -8,10 +8,16 @@ public class BdfError extends RuntimeException
private static final String[] ERRORS = {
"Syntax error",
"End of file",
"Unescaped comment",
"Unescaped string",
"Number format error",
};
public static final int ERROR_SYNTAX = 0;
public static final int ERROR_END_OF_FILE = 1;
public static final int ERROR_UNESCAPED_COMMENT = 2;
public static final int ERROR_UNESCAPED_STRING = 3;
public static final int ERROR_NUMBER = 4;
public static BdfError createError(int errorID, BdfStringPointer ptr)
{
@ -33,13 +39,7 @@ public class BdfError extends RuntimeException
continue;
}
if(array[i] == '\t') {
at = at / 4 * 4 + 4;
}
else {
at += 1;
}
at += 1;
}
int line_size = 0;
@ -47,16 +47,16 @@ public class BdfError extends RuntimeException
for(int i=start_of_line;i<array.length;i++)
{
if(i == array.length - 1) {
break;
}
if(array[i] == '\n') {
break;
}
line_size += 1;
if(i == array.length - 1) {
break;
}
if(i < location)
{
if(array[i] == '\t') {
@ -86,11 +86,17 @@ public class BdfError extends RuntimeException
BdfError bdf_error = new BdfError(message);
bdf_error.error_short = error_short;
bdf_error.type = errorID;
return bdf_error;
}
private String error_short;
private int type;
public int getType() {
return type;
}
public String getErrorShort() {
return error_short;

87
src/bdf/util/FileHelpers.java
View File

@ -1,87 +0,0 @@
package bdf.util;
import java.io.ByteArrayOutputStream;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.zip.GZIPInputStream;
import java.util.zip.InflaterInputStream;
public class FileHelpers
{
public static byte[] readAll(InputStream in)
{
try
{
// Get bytes to return
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
byte[] bytes = new byte[1024];
int size = 0;
while((size = in.read(bytes)) != -1) {
buffer.write(bytes, 0, size);
}
// Send the bytes collected from the file stream back
return buffer.toByteArray();
}
catch (IOException e)
{
// Throw the IOException as a runtime exception
throw new RuntimeException(e);
}
}
public static byte[] readAll(String path) throws IOException
{
// Create the file input stream
FileInputStream in = new FileInputStream(path);
// Load all of its data
byte[] data = readAll(in);
// Close the file input stream
in.close();
// Send back the data
return data;
}
public static byte[] readAllCompressed(String path) throws IOException
{
// Create the file input stream
GZIPInputStream in = new GZIPInputStream(new FileInputStream(path));
// Load all of its data
byte[] data = readAll(in);
// Close the file input stream
in.close();
// Send back the data
return data;
}
public static byte[] readAllIgnoreErrors(String path)
{
try {
return readAll(path);
}
catch(IOException e) {
return new byte[0];
}
}
public static byte[] readAllCompressedIgnoreErrors(String path)
{
try {
return readAllCompressed(path);
}
catch(IOException | RuntimeException e) {
return new byte[0];
}
}
}

89
src/tests/Tests.java
View File

@ -1,57 +1,13 @@
package tests;
import java.io.FileOutputStream;
import java.io.FileInputStream;
import java.io.IOException;
import bdf.classes.IBdfClassManager;
import bdf.data.IBdfDatabase;
import bdf.file.BdfFileManager;
import bdf.types.BdfArray;
import bdf.types.BdfIndent;
import bdf.types.BdfNamedList;
import bdf.types.BdfObject;
import bdf.types.BdfReader;
import bdf.types.BdfTypes;
import bdf.util.FileHelpers;
public class Tests
{
private static class Storage implements IBdfClassManager
{
String name;
int type;
int age;
public Storage(String name, int age, int type) {
this.name = name;
this.age = age;
this.type = type;
}
@Override
public void BdfClassLoad(BdfObject bdf)
{
}
@Override
public void BdfClassSave(BdfObject bdf)
{
BdfNamedList nl = bdf.getNamedList();
nl.set("name", bdf.newObject().setString(name));
if(age != -1) {
nl.set("age", bdf.newObject().setAutoInt(age));
}
if(type != -1) {
nl.set("type", bdf.newObject().setAutoInt(type));
}
}
}
public static void displayHex(IBdfDatabase db)
{
char[] hex_chars = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'};
@ -73,37 +29,28 @@ public class Tests
public static void main(String[] args) throws IOException
{
BdfReader reader = new BdfReader();
BdfObject bdf = reader.getObject();
@SuppressWarnings("resource")
FileInputStream rand = new FileInputStream("/dev/urandom");
byte[] buffer = new byte[100];
bdf.getKeyLocation("type");
bdf.getKeyLocation("qwerty");
bdf.getKeyLocation("age");
bdf.getKeyLocation("name");
long start = System.currentTimeMillis();
long done = 0;
int i = 0;
Storage[] items = {
new Storage("test1", -1, 1),
new Storage("test2", 69, 2),
new Storage("test3", 420, -1),
new Storage("test4", 23, 3),
new Storage("test5", -1, -1),
};
for(;;)
{
if(System.currentTimeMillis() - start > 1000) {
System.out.println("" + i + ": " + done);
start += 1000;
done = 0;
i += 1;
}
BdfArray array = bdf.newArray(5);
bdf.setArray(array);
done += 1;
for(int i=0;i<items.length;i++) {
items[i].BdfClassSave(array.get(i));
rand.read(buffer);
new BdfReader(buffer);
}
reader.serializeHumanReadable(System.out);
IBdfDatabase db = reader.serialize();
displayHex(db);
reader = new BdfReader(db);
reader.serializeHumanReadable(System.out);
}
}