Compare commits

...

33 Commits
v1.0 ... master

Author SHA1 Message Date
josua a75f208b02 Fixed an issue with the README, improved error messages 2020-09-13 16:17:53 +10:00
Josua Robson 87d916b0b4
Merge pull request from jsrobson10/dev
Merge dev into master
2020-09-09 14:06:35 +10:00
jsrobson10 960d0c3b17 Updated the README, added support for non-real integers, fixed issues
when trying to parse garbage data, cleaned up some code.
2020-09-09 14:02:38 +10:00
jsrobson10 04657bc6ad Updated gitignore 2020-08-19 13:40:15 +10:00
josua 01348a5adf Got something working 2020-08-18 22:52:08 +10:00
josua 89426bff98 Added human readable de-serialization, modified the human readable
layout slightly to allow for empty primitive arrays, added error
handling for human readable data, started working on README
2020-08-17 22:54:22 +10:00
josua 8ba902bacd Trying to compress the format as much as possible 2020-08-14 12:25:04 +10:00
josua f73eb0454f Got a functional version of BDF 3 2020-07-27 11:11:01 +10:00
jsrobson10 31e6b9b637 Added emoji support 2020-07-24 17:54:56 +10:00
josua b62c43e892 Started working on a lookup table system for BdfNamedList to save memory and space 2020-07-24 11:26:52 +10:00
jsrobson10 42374c79c5
Added implementation details to the README 2020-07-22 15:48:08 +10:00
jsrobson10 471fbfe42b
Update README.md 2020-07-22 15:27:04 +10:00
jsrobson10 83778d8379
Update README.md 2020-07-22 15:25:49 +10:00
jsrobson10 7a50410132 Moved the compressed variable set from a static method to the constructor 2020-07-22 11:34:03 +10:00
josua 1c803c6125 Forgot to store compression flag to BdfFileManager 2020-07-22 11:30:45 +10:00
josua 984c78020a Added the compressed file manager functionality to the BdfFileManager
class and fixed issues with boolean.
2020-07-21 21:17:51 +10:00
jsrobson10 23a49277c1 Modified the human readable serialization to use streams instead of
strings, to make human readable formats able to handle an output of
greater than 2GB.
2020-06-20 13:38:59 +10:00
josua 39de6a5df4 Improved efficiency of the bdf format by implementing a "pointer"
solution. Fixed the issue with very large files taking up too much
memory. Made the BdfDatabase object more useful in general.
2020-06-18 22:54:27 +10:00
jsrobson10 03e4b583f4 Fixed some issues with indenting 2020-03-24 10:11:32 +11:00
jsrobson10 47e7822177 Added indentation support to the human readable serialisation 2020-03-24 09:39:56 +11:00
jsrobson10 a05cbe7375 Added support for compression 2020-03-19 14:17:55 +11:00
josua 376e5c5261 Added arrays for all the primitives 2020-02-27 13:01:04 +11:00
josua 8ed682653e Added the ByteBuffer type 2019-09-21 13:27:56 +10:00
josua 5a471d0f8c Fixed some issues with iteration, updated readme 2019-07-26 20:07:05 +10:00
josua bfafa4658e Cleaned the code, updated README, made the structure more forgiving 2019-07-26 12:25:16 +10:00
jsrobson10 3b94ac2646
Merge pull request from jsrobson10/save
Save
2019-07-10 14:42:09 +10:00
jsrobson10 7fd421d2b8 Added some more information to the README 2019-07-10 12:57:01 +10:00
jsrobson10 5d6285435a Added information to the README about saving classes 2019-07-10 12:54:10 +10:00
jsrobson10 269ab3a386 Added lots of helper functions to BdfNamedList and BdfObject 2019-07-10 12:37:51 +10:00
jsrobson10 8ca912589b Added lots of helper functions to BdfObject and BdfNamedList 2019-07-10 12:37:21 +10:00
jsrobson10 da3b6a117d Changed BdfNamedList in IBdfClassManager to BdfObject 2019-07-10 12:15:25 +10:00
jsrobson10 ad7d8cb04a Made some modifications to the README 2019-07-10 12:02:36 +10:00
jsrobson10 f01696328b Added some functionality to allow saving classes easier 2019-07-10 12:00:39 +10:00
25 changed files with 3055 additions and 609 deletions

2
.gitignore vendored
View File

@ -1 +1,3 @@
/bin/ /bin/
/*.bdf
/*.hbdf

332
README.md
View File

@ -3,23 +3,28 @@
### Links ### Links
- <a href="#overview">Overview</a> - <a href="#overview">Overview</a>
- <a href="#languages">Languages</a>
- <a href="#data-types">Data types</a> - <a href="#data-types">Data types</a>
- <a href="#creating-an-object">Creating an object</a> - <a href="#creating-an-object">Creating an object</a>
- <a href="#arrays">Arrays</a> - <a href="#arrays">Arrays</a>
- <a href="#named-lists">Named lists</a> - <a href="#named-lists">Named lists</a>
- <a href="#example-bdf-program">Example BDF program</a> - <a href="#human-readable-representation">Human readable representation</a>
- <a href="#special-notes">Special notes</a>
### Overview ### Overview
Binary Data Format (or BDF) is designed to store data in a tag-like binary structure, Binary Data Format (BDF) is a statically typed data representation
like Notch's NBT format, but also open source and free like JSON. The format is format. It was made to be free, fast, compact, and seamlessly
fast and allows multiple data types, it uses 32-bit integers, so BDF files can convertable between its human readable and binary representations.
be fast and work well on 32-bit systems, but have a maximum size of 2 GB.
BDF allows human readable serialization to see what is going on for debugging ### Languages
purposes, but it currently can't parse the result to an object.
- <a href="https://github.com/jsrobson10/BdfCpp">C++</a>
- Java
### Data types ### Data types
- Undefined
- Boolean - Boolean
- Integer - Integer
- Long - Long
@ -27,20 +32,33 @@ purposes, but it currently can't parse the result to an object.
- Byte - Byte
- Double - Double
- Float - Float
- Boolean Array
- Integer Array
- Long Array
- Short Array
- Byte Array
- Double Array
- Float Array
- String - String
- Array - Array
- Named List - Named List
- Empty
### Creating an object ### Creating an object
You will need to create a new object to store any data, use a `BdfObject` instance. You will need to generate a BdfObject to serialize anything,
You can input serialized data into `BdfObject` via a `BdfDatabase`. this can be done by first generating a BdfReader, or generating
a new object via an existing BdfObject.
```java ```java
// New BDF object // Create a reader object
BdfObject bdf = new BdfObject(); BdfReader reader = new BdfReader();
// Get the BdfObject instance
BdfObject bdf = reader.getObject();
// Generate another BdfObject instance
BdfObject bdf_new = bdf.newObject();
// Get an integer // Get an integer
int v = bdf.getInteger(); int v = bdf.getInteger();
@ -48,6 +66,16 @@ int v = bdf.getInteger();
// Set an integer // Set an integer
bdf.setInteger(5); bdf.setInteger(5);
// Set an integer with an automatic type
bdf.setAutoInt(53);
// Set a primitive array of ints
int intArray[] = {3, 4, 5, 6};
bdf.setIntegerArray(intArray);
// Get a byte array
byte[] byteArray = bdf.getByteArray();
// Get the type of variable of the object // Get the type of variable of the object
int type = bdf.getType(); int type = bdf.getType();
@ -58,178 +86,216 @@ if(type == BdfTypes.INTEGER)
} }
// Serialize the BDF object // Serialize the BDF object
byte[] data = bdf.serialize().getBytes(); byte[] data = bdf->serialize(&data, &data_size);
// Load another reader object from the serialized bytes
BdfReader reader2 = new BdfReader(data);
/*
A reader object can be serialized to the human readable
representation as a string or sent over a stream
*/
reader2.serializeHumanReadable(System.out, new BdfIndent("\t", "\n"));
String data_hr = reader2.serializeHumanReadable(new BdfIndent("\t", "\n"));
// Load another BDF object with the serialized bytes // A reader object can be loaded from a human readable object
BdfObject bdf2 = new BdfObject(new BdfDatabase(data)); BdfReader reader3 = new BdfReaderHuman(data_hr);
```
A `BdfFileManager`
instance can be used in the same way as a `BdfObject`, but it also needs a String parameter
for the path of the file. The file can be written with `BdfFileManager.saveDatabase()`.
A `BdfFileManager` is an instance of `BdfObject`, a `BdfFileManager` can be casted to
a `BdfObject`.
```java
// Open a file
BdfFileManager bdf = new BdfFileManager("file.bdf");
// Save the database
bdf.saveDatabase();
// The file can be casted to a BdfObject
BdfObject bdf2 = (BdfObject) bdf;
// Bdf
System.out.println(bdf instanceof BdfObject); // true
``` ```
### Arrays ### Arrays
Arrays can be used to store lists of information, they hold `BdfObject`. Arrays can be used to store chunks of information, they hold instances of
The array is called with `new BdfArray()`. It can hold information, get BdfObject. Arrays can also be iterated over just like any other array.
the size of the array with `BdfArray.size()`, remove elements with
`BdfArray.remove(index)`, set indexes with `BdfArray.set(index, BdfObject)`,
and add elements with `BdfArray.add(BdfObject)`. Arrays also
have support for Iterators and are an instance of `Iterable`.
```java ```java
// New BDF Object BdfReader reader = new BdfReader();
BdfObject bdf = new BdfObject(); BdfObject bdf = reader.getObject();
// New BDF Array // Can be created from a bdf object
BdfArray array = new BdfArray(); BdfArray array = bdf.newArray();
// Size // Get the length of an array
int size = array.size(); int size = array.size();
// Remove // Remove any index from an array
array.remove(3); // Could be any number array.remove(3);
// Set - Could be any number with any object // Set an object to an index of an array
array.set(4, BdfObject.with("A String")); array.set(4, bdf.newObject().setString("A String"));
// Add - Could be any object // Add an object to an array
array.add(BdfObject.with(53)); array.add(bdf.newObject().setByte((byte)53));
// Set the array to the bdf object // Set the array to the bdf object
bdf.setArray(array); bdf.setArray(array);
// Iterate over an array
for(BdfObject o : array)
{
}
``` ```
### Named lists ### Named lists
Named lists can be used to store data under strings, Named lists can be used to store data under ids/strings
to be used like variables in a program. A named list to be used like variables in a program. A named list
can be created with `new BdfNamedList()` and it can be created similar to an array.
has the ability to set with `BdfNamedList.set(String, BdfObject)`,
remove with `BdfNamedList.remove(String)`, and check
for a key with `BdfNamedList.contains(String)`. It also has
features to get all the keys with `BdfNamedList.getKeys()`.
Named lists also have Iterator support and are an instance of
`Iterable`.
```java ```java
// New bdf named list BdfReader reader = new BdfReader();
BdfNamedList list = new BdfNamedList(); BdfObject bdf = reader.getObject();
// Set an element with a value // New named list
list.set("key1", BdfObject.with(5)); BdfNamedList list = bdf.newNamedList();
// Set an element to the named list
list.set("key1", bdf.newObject().setInteger(5));
// Use ids instead of strings for optimisation
// if set/get is being called multiple times
// on the same key.
int key2 = bdf.getKeyLocation("key2");
list.set(key2, bdf.newObject().setFloat(42.0F));
// Get an elements value // Get an elements value
int v = list.get("key1").getInteger(); int v = list.get("key1").getInteger();
// Check if an element exists // Check if an element exists
boolean has_key = list.contains("key1"); bool has_key = list.contains("key1");
// Get the lists keys // Get the lists keys
String[] keys = list.getKeys(); int[] keys = list.getKeys();
// Iterate over the lists keys // Iterate over the lists keys
for(String key : list) for(int key : keys)
{ {
// Get the keys name
String key_name = bdf.getKeyName(key);
} }
``` ```
### Human readable representation
### Example BDF program A big part of binary data format is the human readable
representation. It has a JSON-like syntax.
This can be used with config files and to modify/view
binaries. A big advantage to using the human readable
representation in configuration files is its support
for comments.
```hbdf
/*
A Named List is represented
by an opening tag and a closing
tag { }
*/
{
/*
A key value pair can be stored
within a Named List with a string
property
*/
"hello": "world",
/*
Integers can be stored here too.
They have a character at the end
to say what type they are.
The tag at the end can be:
- I: Integer - a value between -2^31 and 2^31 - 1
- S: Short - a value between -32768 and 32767
- L: Long - a value between -2^63 and 2^63 - 1
- B: Byte - a value between -128 and 127
- D: Double - has 15 decimal digits of precision
- F: Float - has 7 decimal digits of precision
*/
"number": 42I,
"byte": -23B,
"decimal": 73.5D,
/*
This is a boolean. It can
be true or false.
*/
"boolTest": false,
/*
Primitive arrays are represented
by a type, an opening tag, and a
closing tag. They are like an array
but they contain only 1 data type.
The tag at the start can be:
- int
- short
- long
- byte
- double
- float
- bool
*/
"intArray": int (
64I, 42I, 63I,
22I, 96I, -12I,
),
/*
The double and float types support
Infinity, -Infinity, and NaN.
They also support both really
high and really low value numbers.
*/
"doubleArray": double (
42.5D, -20D, 400D,
NaND, -InfinityD, InfinityD,
5.3e-200D, 4e+120D, 2.2e200D,
),
/*
Arrays are enclosed by an opening
tag and a closing tag [ ]
Like the Named List, it can hold
any data type.
*/
"people": [
{"name": "foo", "age": 60B},
{"name": "bar", "age": 21B},
],
// This is a single-line comment
/* This is a multi-line comment */
}
```
### Special notes
Don't mix bdf types between different
readers, this will cause problems.
```java ```java
// Create a new BdfObject instance BdfReader reader1 = new BdfReader();
BdfObject bdf = new BdfObject(); BdfReader reader2 = new BdfReader();
// Create a named list BdfObject bdf1 = reader1.getObject();
BdfNamedList bdf_nl = new BdfNamedList(); BdfObject bdf2 = reader2.getObject();
// Add some variables to the named list // Don't do this
bdf_nl.set("boolean", BdfObject.with(true)); bdf1.setNamedList(bdf2.newNamedList());
bdf_nl.set("an_int", BdfObject.with((int)53));
bdf_nl.set("double", new BdfObject().setDouble(632.5));
// Output some checks on BdfNamedList // Or this
System.out.println(bdf_nl.contains("an_int")); // true bdf1.setArray(bdf2.newArray());
System.out.println(bdf_nl.contains("this_dosn't_exist")); // false
// Create an array // Or this
BdfArray bdf_array = new BdfArray(); BdfNamedList nl = bdf1.newArray();
nl.set("illegal", bdf2.newObject().setString("action"));
// Add some values to the array ```
bdf_array.add(BdfObject.with("Hello, World!"));
bdf_array.add(BdfObject.with(1234567890L));
bdf_array.set(1, BdfObject.with((short)432));
// Output the size of the array
System.out.println(bdf_array.size()); // 2
// Output the type of the 2nd item in the array, value types are in BdfTypes
System.out.println(bdf_array.get(1).getType()); // 3 (BdfTypes.SHORT)
// Save the array to the named list
bdf_nl.set("array", BdfObject.with(bdf_array));
// Set the named list to the bdf object
bdf.setNamedList(bdf_nl);
// Serialize the data
byte[] bdf_data = bdf.serialize().getBytes();
// Load the serialized data
BdfObject bdf2 = new BdfObject(new BdfDatabase(bdf_data));
// Show the human readable serialized data
System.out.println(bdf2.serializeHumanReadable()); // {"boolean": true, "an_int": 53I, "double": 632.5D, "array": ["Hello, World!", 432S]}
// Show the value of the boolean in the named list
System.out.println(bdf2.getNamedList().get("boolean").getBoolean()); // true
// Show the value of item 0 in the array
System.out.println(bdf2.getNamedList().get("array").getArray().get(0).getString()); // Hello, World!
// Check if the double exists
System.out.println(bdf2.getNamedList().contains("double")); // true
// Remove the double from the named list
bdf2.getNamedList().remove("double");
// Check if the double exists
System.out.println(bdf2.getNamedList().contains("double")); // false
```

Binary file not shown.

View File

@ -1,75 +0,0 @@
package bdf;
import bdf.data.BdfDatabase;
import bdf.types.BdfArray;
import bdf.types.BdfNamedList;
import bdf.types.BdfObject;
public class Tests {
public static void main(String[] args)
{
// Create a new BdfObject instance
BdfObject bdf = new BdfObject();
// Create a named list
BdfNamedList bdf_nl = new BdfNamedList();
// Add some variables to the named list
bdf_nl.set("boolean", BdfObject.with(true));
bdf_nl.set("an_int", BdfObject.with((int)53));
bdf_nl.set("double", new BdfObject().setDouble(632.5));
// Output some checks on BdfNamedList
System.out.println(bdf_nl.contains("an_int")); // true
System.out.println(bdf_nl.contains("this_dosn't_exist")); // false
// Create an array
BdfArray bdf_array = new BdfArray();
// Add some values to the array
bdf_array.add(BdfObject.with("Hello, World!"));
bdf_array.add(BdfObject.with(1234567890L));
bdf_array.set(1, BdfObject.with((short)432));
// Output the size of the array
System.out.println(bdf_array.size()); // 2
// Output the type of the 2nd item in the array, value types are in BdfTypes
System.out.println(bdf_array.get(1).getType()); // 3 (BdfTypes.SHORT)
// Save the array to the named list
bdf_nl.set("array", BdfObject.with(bdf_array));
// Set the named list to the bdf object
bdf.setNamedList(bdf_nl);
// Serialize the data
byte[] bdf_data = bdf.serialize().getBytes();
// Load the serialized data
BdfObject bdf2 = new BdfObject(new BdfDatabase(bdf_data));
// Show the human readable serialized data
System.out.println(bdf2.serializeHumanReadable()); // {"boolean": true, "an_int": 53I, "double": 632.5D, "array": ["Hello, World!", 432S]}
// Show the value of the boolean in the named list
System.out.println(bdf2.getNamedList().get("boolean").getBoolean()); // true
// Show the value of item 0 in the array
System.out.println(bdf2.getNamedList().get("array").getArray().get(0).getString()); // Hello, World!
// Check if the double exists
System.out.println(bdf2.getNamedList().contains("double")); // true
// Remove the double from the named list
bdf2.getNamedList().remove("double");
// Check if the double exists
System.out.println(bdf2.getNamedList().contains("double")); // false
}
}

View File

@ -1,62 +1,170 @@
package bdf.data; package bdf.data;
import java.nio.charset.StandardCharsets; import java.io.IOException;
import java.io.OutputStream;
public class BdfDatabase public class BdfDatabase implements IBdfDatabase
{ {
protected byte[] database = null; static final int STREAM_CHUNK_SIZE = 1024*1024;
public BdfDatabase() { byte[] database;
this.database = new byte[0];
public BdfDatabase(byte[] bytes) {
database = bytes;
} }
public BdfDatabase(String database) { public BdfDatabase(String str) {
this.database = database.getBytes(); this(str.getBytes());
} }
public BdfDatabase(byte ...database) { public BdfDatabase(int size) {
this.database = database; this.database = new byte[size];
} }
public BdfDatabase getAt(int start, int end) @Override
public IBdfDatabase getCopy(int start, int end)
{ {
byte[] split = new byte[end - start]; byte[] database = new byte[end - start];
for(int i=start;i<end;i++) { for(int i=0;i<end-start;i++) {
split[i-start] = this.database[i]; database[i] = this.database[i + start];
} }
return new BdfDatabase(split); return new BdfDatabase(database);
} }
public int length() { @Override
return this.database.length; public byte getByte() {
} return database[0];
public byte[] getBytes() {
return this.database;
} }
@Override
public byte getByte(int i) { public byte getByte(int i) {
return this.database[i]; return database[i];
} }
public String getString() { @Override
return new String(this.database, StandardCharsets.UTF_8); public byte[] getBytes(int start, int size)
}
public static BdfDatabase add(BdfDatabase d1, BdfDatabase d2)
{ {
byte[] added = new byte[d1.length() + d2.length()]; byte[] database = new byte[size];
for(int i=0;i<d1.length();i++) { for(int i=0;i<size;i++) {
added[i] = d1.getByte(i); database[i] = this.database[i + start];
} }
for(int i=0;i<d2.length();i++) { return database;
added[d1.length()+i] = d2.getByte(i); }
@Override
public byte[] getBytes() {
return getBytes(0, database.length);
}
@Override
public IBdfDatabase getPointer(int location, int size)
{
BdfDatabasePointer db = new BdfDatabasePointer();
db.database = database;
db.location = location;
db.size = size;
return db;
}
@Override
public IBdfDatabase getPointer(int location) {
return getPointer(location, location);
}
@Override
public int size() {
return database.length;
}
@Override
public String getString() {
return new String(database);
}
@Override
public void writeToStream(OutputStream stream, int start, int size) throws IOException
{
for(int i=0;i<size;i+=STREAM_CHUNK_SIZE)
{
if(size - i < STREAM_CHUNK_SIZE) {
stream.write(getBytes(i + start, size - i));
} else {
stream.write(getBytes(i + start, STREAM_CHUNK_SIZE));
}
}
}
@Override
public void writeToStream(OutputStream stream) throws IOException {
writeToStream(stream, 0, database.length);
}
public static IBdfDatabase add(IBdfDatabase b1, IBdfDatabase b2)
{
byte[] bytes = new byte[b1.size() + b2.size()];
int b1_size = b1.size();
for(int i=0;i<bytes.length;i++)
{
if(i >= b1_size) {
bytes[i] = b2.getByte(i - b1_size);
} else {
bytes[i] = b1.getByte(i);
}
} }
return new BdfDatabase(added); return new BdfDatabase(bytes);
}
@Override
public void setByte(int pos, byte b) {
database[pos] = b;
}
@Override
public void setByte(byte b) {
database[0] = b;
}
@Override
public void setBytes(IBdfDatabase bytes) {
setBytes(bytes, 0, bytes.size());
}
@Override
public void setBytes(IBdfDatabase bytes, int offset) {
setBytes(bytes, offset, bytes.size());
}
@Override
public void setBytes(IBdfDatabase bytes, int offset, int length)
{
for(int i=0;i<length;i++) {
database[offset + i] = bytes.getByte(i);
}
}
@Override
public void setBytes(byte[] bytes, int offset, int length)
{
for(int i=0;i<length;i++) {
database[offset + i] = bytes[i];
}
}
@Override
public void setBytes(byte[] bytes, int offset) {
setBytes(bytes, offset, bytes.length);
}
@Override
public void setBytes(byte[] bytes) {
setBytes(bytes, 0, bytes.length);
} }
} }

View File

@ -0,0 +1,160 @@
package bdf.data;
import java.io.IOException;
import java.io.OutputStream;
public class BdfDatabasePointer implements IBdfDatabase
{
byte[] database;
int location;
int size;
public BdfDatabasePointer(byte[] bytes) {
database = bytes;
size = bytes.length;
location = 0;
}
public BdfDatabasePointer(String str) {
this(str.getBytes());
}
public BdfDatabasePointer(int size) {
this.database = new byte[size];
this.location = 0;
this.size = size;
}
BdfDatabasePointer() {
}
@Override
public IBdfDatabase getCopy(int start, int end)
{
byte[] database = new byte[end - start];
for(int i=0;i<end-start;i++) {
database[i] = this.database[i + start + location];
}
return new BdfDatabase(database);
}
@Override
public byte getByte() {
return database[location];
}
@Override
public byte getByte(int i) {
return database[location + i];
}
@Override
public byte[] getBytes(int start, int size)
{
byte[] database = new byte[size];
for(int i=0;i<size;i++) {
database[i] = this.database[i + location + start];
}
return database;
}
@Override
public byte[] getBytes() {
return getBytes(0, size);
}
@Override
public IBdfDatabase getPointer(int location, int size)
{
BdfDatabasePointer db = new BdfDatabasePointer();
db.database = database;
db.location = this.location + location;
db.size = size;
return db;
}
@Override
public IBdfDatabase getPointer(int location) {
return getPointer(location, size - location);
}
@Override
public int size() {
return size;
}
@Override
public String getString() {
return new String(database, location, size);
}
@Override
public void writeToStream(OutputStream stream, int start, int size) throws IOException
{
for(int i=0;i<size;i+=BdfDatabase.STREAM_CHUNK_SIZE)
{
if(size - i < BdfDatabase.STREAM_CHUNK_SIZE) {
stream.write(getBytes(i + start, size - i));
} else {
stream.write(getBytes(i + start, BdfDatabase.STREAM_CHUNK_SIZE));
}
}
}
@Override
public void writeToStream(OutputStream stream) throws IOException {
writeToStream(stream, 0, size);
}
@Override
public void setByte(int pos, byte b) {
database[pos + location] = b;
}
@Override
public void setByte(byte b) {
database[location] = b;
}
@Override
public void setBytes(IBdfDatabase bytes) {
setBytes(bytes, 0, bytes.size());
}
@Override
public void setBytes(IBdfDatabase bytes, int offset) {
setBytes(bytes, offset, bytes.size());
}
@Override
public void setBytes(IBdfDatabase bytes, int offset, int length)
{
for(int i=0;i<length;i++) {
database[offset + location + i] = bytes.getByte(i);
}
}
@Override
public void setBytes(byte[] bytes, int offset, int length)
{
for(int i=0;i<length;i++) {
database[offset + location + i] = bytes[i];
}
}
@Override
public void setBytes(byte[] bytes, int offset) {
setBytes(bytes, offset, bytes.length);
}
@Override
public void setBytes(byte[] bytes) {
setBytes(bytes, 0, bytes.length);
}
}

View File

@ -0,0 +1,290 @@
package bdf.data;
import bdf.util.BdfError;
public class BdfStringPointer
{
char[] data;
int offset;
public BdfStringPointer(char[] data, int offset) {
this.data = data;
this.offset = offset;
}
public BdfStringPointer getPointer(int offset) {
return new BdfStringPointer(data, this.offset + offset);
}
public void increment(int amount) {
offset += amount;
}
public void increment() {
offset += 1;
}
public char[] getDataCharArray() {
return data;
}
public int getDataLocation() {
return offset;
}
public int getDataLength() {
return data.length;
}
public char[] getCharArray(int offset, int length)
{
char[] array = new char[length];
for(int i=0;i<length;i++) {
array[i] = data[i + offset + this.offset];
}
return array;
}
public char[] getCharArray(int length) {
return getCharArray(0, length);
}
public char getChar(int i) {
return data[offset + i];
}
public char getChar() {
return data[offset];
}
public void ignoreBlanks()
{
for(;;)
{
if(offset >= data.length) {
throw BdfError.createError(BdfError.ERROR_END_OF_FILE, this);
}
char c = getChar();
// Comments
if(c == '/' && offset < data.length)
{
char c2 = getChar(1);
// Line comment
if(c2 == '/')
{
for(;;)
{
if(offset + 1 >= data.length) {
break;
}
increment();
c = getChar();
if(c == '\n') {
break;
}
}
}
// Multi-line comment
else if(c2 == '*')
{
for(;;)
{
if(offset + 1 >= data.length) {
throw BdfError.createError(BdfError.ERROR_UNESCAPED_COMMENT, getPointer(-1));
}
increment();
c = getChar();
if(c == '*' && offset < data.length && getChar(1) == '/') {
increment();
break;
}
}
}
else {
return;
}
}
else if(!(c == '\n' || c == '\t' || c == ' ')) {
return;
}
increment();
}
}
// In the format "abc\n\t\u0003..."
public String getQuotedString()
{
if(getChar() != '"') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, this);
}
increment();
String str = "";
for(;;)
{
if(offset >= data.length) {
throw BdfError.createError(BdfError.ERROR_UNESCAPED_STRING, this);
}
char c = getChar();
// Check for back slashes
if(c == '\\')
{
increment();
c = getChar();
switch(c)
{
case 'n':
str += "\n";
increment();
break;
case 't':
str += "\t";
increment();
break;
case '"':
str += "\"";
increment();
break;
case '\\':
str += "\\";
increment();
break;
case '\n':
str += "\n";
increment();
break;
case 'u': // \u0000
{
if(offset + 5 >= data.length) {
throw BdfError.createError(BdfError.ERROR_UNESCAPED_STRING, getPointer(1));
}
char[] hex = getCharArray(1, 4);
char unicode = (char)0;
int m = 1;
for(int j=hex.length-1;j>=0;j--)
{
c = hex[j];
if(c >= '0' && c <= '9') {
unicode += (char)(m * (c - '0'));
}
else if(c >= 'a' && c <= 'f') {
unicode += (char)(m * (c - 'a' + 10));
}
else {
throw BdfError.createError(BdfError.ERROR_SYNTAX, getPointer(1 + (hex.length-j-1)));
}
m *= 16;
}
str += unicode;
increment(5);
break;
}
default:
str += "\\" + c;
}
}
else if(c == '\n') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, this);
}
else if(c == '"') {
increment();
break;
}
else {
increment();
str += c;
}
}
return str;
}
public boolean isNext(String check)
{
if(check.length() + offset > data.length) {
return false;
}
for(int i=0;i<check.length();i++)
{
char c = getChar(i);
c = (char)((c >= 'A' && c <= 'Z') ? (c + 32) : c);
if(c != check.charAt(i)) {
return false;
}
}
increment(check.length());
return true;
}
public boolean isInteger()
{
for(int i=offset;i<data.length;i++)
{
char c = data[i];
switch(c)
{
case 'I':
return true;
case 'L':
return true;
case 'S':
return true;
case 'B':
return true;
case 'D':
return false;
case 'F':
return false;
case 'e':
continue;
case 'E':
continue;
case '.':
continue;
case '-':
continue;
}
if(c >= '0' && c <= '9') {
continue;
}
throw BdfError.createError(BdfError.ERROR_SYNTAX, new BdfStringPointer(data, i));
}
throw BdfError.createError(BdfError.ERROR_END_OF_FILE, new BdfStringPointer(data, data.length - 1));
}
}

View File

@ -0,0 +1,34 @@
package bdf.data;
import java.io.IOException;
import java.io.OutputStream;
public interface IBdfDatabase
{
public IBdfDatabase getCopy(int start, int end);
public IBdfDatabase getPointer(int location, int size);
public IBdfDatabase getPointer(int location);
public void writeToStream(OutputStream stream, int start, int size) throws IOException;
public void writeToStream(OutputStream stream) throws IOException;
public int size();
public byte[] getBytes();
public byte[] getBytes(int start, int size);
public byte getByte();
public byte getByte(int i);
public String getString();
public void setBytes(byte[] bytes, int offset, int length);
public void setBytes(byte[] bytes, int offset);
public void setBytes(byte[] bytes);
public void setBytes(IBdfDatabase bytes, int offset, int length);
public void setBytes(IBdfDatabase bytes, int offset);
public void setBytes(IBdfDatabase bytes);
public void setByte(int pos, byte b);
public void setByte(byte b);
}

View File

@ -1 +0,0 @@
/UndefinedKeyException.class

View File

@ -1,11 +0,0 @@
package bdf.exception;
public class UndefinedKeyException extends RuntimeException
{
private static final long serialVersionUID = 1L;
public UndefinedKeyException(String key)
{
super("The key \""+key+"\" has not been defined.");
}
}

View File

@ -1 +0,0 @@
/BdfFileManager.class

View File

@ -1,64 +0,0 @@
package bdf.file;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import bdf.data.BdfDatabase;
import bdf.types.BdfObject;
import bdf.util.FileHelpers;
public class BdfFileManager extends BdfObject
{
protected String path;
private static BdfDatabase init(String path)
{
// Get the file
File file = new File(path);
// Does the file have read access
if(file.canRead())
{
// Return the files contents as a database
return new BdfDatabase(FileHelpers.readAllIgnoreErrors(path));
}
// Return an empty database if there is no read access
return new BdfDatabase();
}
public BdfFileManager(String path) {
super(init(path));
this.path = path;
}
public void saveDatabase(String path)
{
try
{
// Get the file handler
File file = new File(path);
// Create the parent directories
file.getParentFile().mkdirs();
// Get the database file for output
FileOutputStream out = new FileOutputStream(path);
// Write the database to the file
out.write(this.serialize().getBytes());
// Close the file output stream
out.close();
}
catch(IOException e) {
return;
}
}
public void saveDatabase() {
this.saveDatabase(this.path);
}
}

View File

@ -1,78 +1,150 @@
package bdf.types; package bdf.types;
import java.io.IOException;
import java.io.OutputStream;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Iterator;
import bdf.data.BdfDatabase; import bdf.data.BdfStringPointer;
import bdf.util.DataHelpers; import bdf.data.IBdfDatabase;
import bdf.util.BdfError;
public class BdfArray implements IBdfType public class BdfArray implements IBdfType, Iterable<BdfObject>
{ {
protected ArrayList<BdfObject> elements = new ArrayList<BdfObject>(); protected ArrayList<BdfObject> elements;
public BdfArray() { BdfArray(BdfLookupTable lookupTable, BdfStringPointer ptr)
{
this.elements = new ArrayList<BdfObject>();
ptr.increment();
// [..., ...]
while(true)
{
ptr.ignoreBlanks();
if(ptr.getChar() == ']') {
ptr.increment();
return;
}
add(new BdfObject(lookupTable, ptr));
// There should be a comma after this
ptr.ignoreBlanks();
char c = ptr.getChar();
if(c == ']') {
ptr.increment();
return;
}
if(c != ',') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
ptr.increment();
ptr.ignoreBlanks();
}
} }
public BdfArray(BdfDatabase data) BdfArray(BdfLookupTable lookupTable, int size)
{ {
this.elements = new ArrayList<BdfObject>(size);
for(int i=0;i<size;i++) {
this.elements.add(new BdfObject(lookupTable));
}
}
BdfArray(BdfLookupTable lookupTable, IBdfDatabase data)
{
this.elements = new ArrayList<BdfObject>();
// Create an iterator value to loop over the data // Create an iterator value to loop over the data
int i = 0; int i = 0;
// Loop over the data // Loop over the data
while(i < data.length()) while(i < data.size())
{ {
// Get the size of the object // Get the size of the object
int size = DataHelpers.getByteBuffer(data.getAt(i, (i+(Integer.SIZE/8)))).getInt(); int size = BdfObject.getSize(data.getPointer(i));
if(size <= 0 || i + size > data.size()) {
return;
}
// Get the object // Get the object
BdfObject object = new BdfObject(data.getAt((i+(Integer.SIZE/8)), (i+(Integer.SIZE/8)+size))); BdfObject object = new BdfObject(lookupTable, data.getPointer(i, size));
// Add the object to the elements list // Add the object to the elements list
elements.add(object); elements.add(object);
// Increase the iterator by the amount of bytes // Increase the iterator by the amount of bytes
i += (Integer.SIZE/8)+size; i += size;
} }
} }
@Override
public BdfDatabase serialize()
{
// Create the serialized data string
BdfDatabase serialized = new BdfDatabase();
// Loop over the elements
for(BdfObject o : elements)
{
// Convert the object to a string
BdfDatabase db = o.serialize();
// Add the serialized object to the serialized data
serialized = BdfDatabase.add(serialized, DataHelpers.serializeInt(db.length()));
serialized = BdfDatabase.add(serialized, db);
}
// Send back the serialized data
return serialized;
}
@Override @Override
public String serializeHumanReadable() public int serializeSeeker(int[] locations)
{ {
String data = "["; int size = 0;
for(BdfObject o : elements) {
size += o.serializeSeeker(locations);
}
return size;
}
@Override
public int serialize(IBdfDatabase database, int[] locations, byte flags)
{
int pos = 0;
for(BdfObject o : elements) {
pos += o.serialize(database.getPointer(pos), locations, (byte)0);
}
return pos;
}
@Override
public void serializeHumanReadable(OutputStream stream, BdfIndent indent, int it) throws IOException
{
if(elements.size() == 0) {
stream.write("[]".getBytes());
return;
}
stream.write('[');
for(int i=0;i<elements.size();i++) for(int i=0;i<elements.size();i++)
{ {
BdfObject o = elements.get(i); BdfObject o = elements.get(i);
data += o.serializeHumanReadable(); stream.write(indent.breaker.getBytes());
if(elements.size() > i+1) for(int n=0;n<=it;n++) {
{ stream.write(indent.indent.getBytes());
data += ", "; }
o.serializeHumanReadable(stream, indent, it + 1);
if(elements.size() > i+1) {
stream.write(", ".getBytes());
} }
} }
return data + "]"; stream.write(indent.breaker.getBytes());
for(int n=0;n<it;n++) {
stream.write(indent.indent.getBytes());
}
stream.write(']');
} }
public BdfArray add(BdfObject o) public BdfArray add(BdfObject o)
@ -91,9 +163,14 @@ public class BdfArray implements IBdfType
return this; return this;
} }
public BdfArray remove(int index) public BdfObject remove(int index) {
{ BdfObject bdf = elements.get(index);
elements.remove(index); elements.remove(index);
return bdf;
}
public BdfArray remove(BdfObject bdf) {
elements.remove(bdf);
return this; return this;
} }
@ -110,4 +187,43 @@ public class BdfArray implements IBdfType
return elements.size(); return elements.size();
} }
@Override
public Iterator<BdfObject> iterator()
{
return new Iterator<BdfObject>()
{
protected int i = 0;
@Override
public boolean hasNext()
{
return elements.size() > i;
}
@Override
public BdfObject next()
{
BdfObject o = elements.get(i);
i++;
return o;
}
@Override
public void remove()
{
i-=1;
elements.remove(i);
}
};
}
@Override
public void getLocationUses(int[] locations) {
for(BdfObject element : elements) {
element.getLocationUses(locations);
}
}
} }

View File

@ -0,0 +1,12 @@
package bdf.types;
public class BdfIndent
{
String indent;
String breaker;
public BdfIndent(String indent, String breaker) {
this.indent = indent;
this.breaker = breaker;
}
}

View File

@ -0,0 +1,139 @@
package bdf.types;
import java.io.OutputStream;
import java.util.ArrayList;
import bdf.data.IBdfDatabase;
import bdf.util.DataHelpers;
class BdfLookupTable implements IBdfType
{
private ArrayList<byte[]> keys;
private BdfReader reader;
BdfLookupTable(BdfReader reader)
{
this.keys = new ArrayList<byte[]>();
this.reader = reader;
}
BdfLookupTable(BdfReader reader, IBdfDatabase database)
{
this.keys = new ArrayList<byte[]>();
this.reader = reader;
for(int i=0;i<database.size();)
{
int key_size = 0xff & database.getByte(i);
i += 1;
if(i + key_size > database.size()) {
return;
}
keys.add(database.getBytes(i, key_size));
i += key_size;
}
}
int getLocation(byte[] name)
{
for(int i=0;i<keys.size();i++)
{
if(DataHelpers.bytesAreEqual(name, keys.get(i))) {
return i;
}
}
keys.add(name);
return keys.size() - 1;
}
boolean hasLocation(int location) {
return location >= 0 && location < keys.size();
}
byte[] getName(int location) {
return keys.get(location);
}
@Override
public int serialize(IBdfDatabase database, int[] locations, byte flags)
{
int upto = 0;
for(int i=0;i<locations.length;i++)
{
int loc = locations[i];
if(loc == -1) {
continue;
}
byte[] key = keys.get(i);
database.setBytes(key, upto + 1);
database.setByte(upto, (byte)key.length);
upto += key.length;
upto += 1;
}
return upto;
}
@Override
public int serializeSeeker(int[] locations)
{
int size = 0;
for(int i=0;i<locations.length;i++)
{
// Skip this key if the location is unset (the key has been culled)
int loc = locations[i];
if(loc == -1) {
continue;
}
size += keys.get(i).length;
size += 1;
}
return size;
}
public int[] serializeGetLocations()
{
int[] locations = new int[keys.size()];
int[] uses = new int[keys.size()];
int next = 0;
reader.bdf.getLocationUses(uses);
for(int i=0;i<locations.length;i++)
{
if(uses[i] > 0) {
locations[i] = next;
next += 1;
} else {
locations[i] = -1;
}
}
return locations;
}
@Override
public void serializeHumanReadable(OutputStream stream, BdfIndent indent, int it) {
}
@Override
public void getLocationUses(int[] locations) {
}
public int size() {
return keys.size();
}
}

View File

@ -1,43 +1,141 @@
package bdf.types; package bdf.types;
import java.nio.charset.StandardCharsets; import java.io.IOException;
import java.io.OutputStream;
import java.nio.ByteBuffer;
import java.util.ArrayList; import java.util.ArrayList;
import bdf.data.BdfDatabase; import bdf.data.BdfStringPointer;
import bdf.exception.UndefinedKeyException; import bdf.data.IBdfDatabase;
import bdf.util.BdfError;
import bdf.util.DataHelpers; import bdf.util.DataHelpers;
public class BdfNamedList implements IBdfType public class BdfNamedList implements IBdfType
{ {
protected class Element protected class Element
{ {
public byte[] key; public int key;
public BdfObject object; public BdfObject object;
} }
protected ArrayList<Element> elements = new ArrayList<Element>(); protected ArrayList<Element> elements = new ArrayList<Element>();
protected BdfLookupTable lookupTable;
public BdfNamedList() { BdfNamedList(BdfLookupTable lookupTable, BdfStringPointer ptr)
{
this.lookupTable = lookupTable;
ptr.increment();
// {"key": ..., "key2": ...}
while(true)
{
ptr.ignoreBlanks();
char c = ptr.getChar();
if(c == '}') {
ptr.increment();
break;
}
if(c != '"') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
String key = ptr.getQuotedString();
// There should be a colon after this
ptr.ignoreBlanks();
if(ptr.getChar() != ':') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
ptr.increment();
ptr.ignoreBlanks();
set(key, new BdfObject(lookupTable, ptr));
// There should be a comma after this
ptr.ignoreBlanks();
c = ptr.getChar();
if(c == '}') {
ptr.increment();
return;
}
if(c != ',') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
ptr.increment();
ptr.ignoreBlanks();
}
}
BdfNamedList(BdfLookupTable lookupTable) {
this.lookupTable = lookupTable;
} }
public BdfNamedList(BdfDatabase data) BdfNamedList(BdfLookupTable lookupTable, IBdfDatabase data)
{ {
this.lookupTable = lookupTable;
// Create an iterator value to loop over the data // Create an iterator value to loop over the data
int i = 0; int i = 0;
// Loop over the data // Loop over the data
while(i < data.length()) while(i < data.size())
{ {
// Get the key
int key_size = DataHelpers.getByteBuffer(data.getAt(i, i+(Integer.SIZE/8))).getInt();
i += (Integer.SIZE/8);
byte[] key = data.getAt(i, i+key_size).getBytes();
// Get the object // Get the object
int key_size = 0;
IBdfDatabase flag_ptr = data.getPointer(i);
int object_size = BdfObject.getSize(flag_ptr);
byte key_size_bytes = BdfObject.getParentFlags(flag_ptr);
if(object_size <= 0 || i + object_size >= data.size()) {
return;
}
BdfObject object = new BdfObject(lookupTable, data.getPointer(i, object_size));
i += object_size;
switch(key_size_bytes)
{
case 2:
key_size = 1;
break;
case 1:
key_size = 2;
break;
case 0:
key_size = 4;
break;
}
// Get the key
ByteBuffer key_buff = DataHelpers.getByteBuffer(data.getPointer(i, key_size));
int key = 0;
switch(key_size_bytes)
{
case 2:
key = 0xff & key_buff.get();
break;
case 1:
key = 0xffff & key_buff.getShort();
break;
case 0:
key = key_buff.getInt();
break;
}
if(!lookupTable.hasLocation(key)) {
return;
}
i += key_size; i += key_size;
int object_size = DataHelpers.getByteBuffer(data.getAt(i, i+(Integer.SIZE/8))).getInt();
i += (Integer.SIZE/8);
BdfObject object = new BdfObject(data.getAt(i, i+object_size));
// Create a new element and save some data to it // Create a new element and save some data to it
Element element = new Element(); Element element = new Element();
@ -46,68 +144,117 @@ public class BdfNamedList implements IBdfType
// Add the object to the elements list // Add the object to the elements list
elements.add(element); elements.add(element);
// Increase the iterator by the amount of bytes
i += object_size;
} }
} }
@Override @Override
public BdfDatabase serialize() public int serialize(IBdfDatabase database, int[] locations, byte flags)
{ {
// Create the serialized data string int pos = 0;
BdfDatabase serialized = new BdfDatabase();
// Loop over the elements
for(Element o : elements) for(Element o : elements)
{ {
// Add the serialized data to the data string int location = locations[o.key];
BdfDatabase data = o.object.serialize();
serialized = BdfDatabase.add(serialized, DataHelpers.serializeInt(o.key.length)); byte size_bytes_tag;
serialized = BdfDatabase.add(serialized, new BdfDatabase(o.key)); byte size_bytes;
serialized = BdfDatabase.add(serialized, DataHelpers.serializeInt(data.length()));
serialized = BdfDatabase.add(serialized, data); if(location > 65535) { // >= 2 ^ 16
size_bytes_tag = 0;
size_bytes = 4;
} else if(location > 255) { // >= 2 ^ 8
size_bytes_tag = 1;
size_bytes = 2;
} else { // < 2 ^ 8
size_bytes_tag = 2;
size_bytes = 1;
}
int size = o.object.serialize(database.getPointer(pos), locations, size_bytes_tag);
int offset = pos + size;
byte[] bytes = DataHelpers.serializeInt(location);
for(int i=0;i<size_bytes;i++) {
database.setByte(i + offset, bytes[i - size_bytes + 4]);
}
pos += size + size_bytes;
} }
// Send back the serialized data return pos;
return serialized;
} }
@Override @Override
public String serializeHumanReadable() public int serializeSeeker(int[] locations)
{ {
String data = "{"; int size = 0;
for(Element o : elements)
{
int location = locations[o.key];
if(location > 65535) { // >= 2 ^ 16
size += 4;
} else if(location > 255) { // >= 2 ^ 8
size += 2;
} else { // < 2 ^ 8
size += 1;
}
size += o.object.serializeSeeker(locations);
}
return size;
}
@Override
public void serializeHumanReadable(OutputStream stream, BdfIndent indent, int it) throws IOException
{
if(elements.size() == 0) {
stream.write("{}".getBytes());
return;
}
stream.write('{');
for(int i=0;i<elements.size();i++) for(int i=0;i<elements.size();i++)
{ {
Element e = elements.get(i); Element e = elements.get(i);
data += DataHelpers.serializeString(new String(e.key, StandardCharsets.UTF_8)); stream.write(indent.breaker.getBytes());
data += ": ";
data += e.object.serializeHumanReadable();
if(elements.size() > i+1) for(int n=0;n<=it;n++) {
{ stream.write(indent.indent.getBytes());
data += ", "; }
stream.write((DataHelpers.serializeString(new String(lookupTable.getName(e.key))) + ": ").getBytes());
e.object.serializeHumanReadable(stream, indent, it + 1);
if(elements.size() > i+1) {
stream.write(", ".getBytes());
} }
} }
return data + "}"; stream.write(indent.breaker.getBytes());
for(int n=0;n<it;n++) {
stream.write(indent.indent.getBytes());
}
stream.write('}');
} }
public BdfObject get(String key) public BdfObject get(int key)
{ {
// Get the object to send back // Get the object to send back
BdfObject object = null; BdfObject object = null;
// Convert the key to bytes
byte[] key_bytes = key.getBytes();
// Loop over the elements // Loop over the elements
for(Element e : elements) for(Element e : elements)
{ {
// Is this the element key // Is this the element key
if(DataHelpers.bytesAreEqual(e.key, key_bytes)) if(e.key == key)
{ {
// Set the object // Set the object
object = e.object; object = e.object;
@ -117,15 +264,18 @@ public class BdfNamedList implements IBdfType
} }
} }
// Raise an error // Get a bdf object
throw new UndefinedKeyException(key); BdfObject o = new BdfObject(lookupTable);
// Set the bdf object
this.set(key, o);
// Send back the object
return o;
} }
public void remove(String key) public BdfObject remove(int key)
{ {
// Convert the key to bytes
byte[] key_bytes = key.getBytes();
// Loop over the elements // Loop over the elements
for(int i=0;i<elements.size();i++) for(int i=0;i<elements.size();i++)
{ {
@ -133,58 +283,50 @@ public class BdfNamedList implements IBdfType
Element e = elements.get(i); Element e = elements.get(i);
// Is the specified key the same as the elements key // Is the specified key the same as the elements key
if(DataHelpers.bytesAreEqual(e.key, key_bytes)) if(e.key == key) {
{ return elements.remove(i).object;
// Delete this element
elements.remove(i);
// Exit out of the function, prevent NullPointException
return;
} }
} }
// Raise an error // Send back nothing
throw new UndefinedKeyException(key); return null;
} }
public void set(String key, BdfObject object) public BdfNamedList set(int key, BdfObject object)
{ {
// Convert the key to bytes
byte[] key_bytes = key.getBytes();
// Loop over the elements, does it already exist // Loop over the elements, does it already exist
for(Element e : elements) for(Element e : elements)
{ {
// Is the key here the same as the specified key // Is the key here the same as the specified key
if(DataHelpers.bytesAreEqual(e.key, key_bytes)) if(e.key == key)
{ {
// Set the new object // Set the new object
e.object = object; e.object = object;
// Exit out of the function, don't add another object // Exit out of the function, don't add another object
return; return this;
} }
} }
// Create a new element object // Create a new element object
Element e = new Element(); Element e = new Element();
e.key = key_bytes;
e.object = object; e.object = object;
e.key = key;
// Add the new element object to the elements list // Add the new element object to the elements list
elements.add(e); elements.add(e);
// Send this class back
return this;
} }
public boolean contains(String key) public boolean contains(int key)
{ {
// Convert the key to bytes
byte[] key_bytes = key.getBytes();
// Loop over the elements // Loop over the elements
for(Element e : elements) for(Element e : elements)
{ {
// Is the elements key the same as the specified key // Is the elements key the same as the specified key
if(DataHelpers.bytesAreEqual(e.key, key_bytes)) if(e.key == key)
{ {
// Send back true to say the element was found // Send back true to say the element was found
return true; return true;
@ -195,25 +337,49 @@ public class BdfNamedList implements IBdfType
return false; return false;
} }
public String[] getKeys() public int[] getKeys()
{ {
// Get the keys to send back // Get the keys to send back
String[] keys = new String[elements.size()]; int[] keys = new int[elements.size()];
// Loop over the elements // Loop over the elements
for(int i=0;i<elements.size();i++) for(int i=0;i<elements.size();i++)
{ {
// Get the element // Get the element
Element e = elements.get(i); Element e = elements.get(i);
keys[i] = new String(e.key, StandardCharsets.UTF_8); keys[i] = e.key;
} }
// Return the list of keys as strings // Return the list of keys as strings
return keys; return keys;
} }
public boolean contains(String key) {
return contains(lookupTable.getLocation(key.getBytes()));
}
public BdfNamedList set(String key, BdfObject object) {
return set(lookupTable.getLocation(key.getBytes()), object);
}
public BdfObject remove(String key) {
return remove(lookupTable.getLocation(key.getBytes()));
}
public BdfObject get(String key) {
return get(lookupTable.getLocation(key.getBytes()));
}
public int size() { public int size() {
return elements.size(); return elements.size();
} }
@Override
public void getLocationUses(int[] locations)
{
for(Element e : elements) {
locations[e.key] += 1;
e.object.getLocationUses(locations);
}
}
} }

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,182 @@
package bdf.types;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.ByteBuffer;
import bdf.data.BdfDatabase;
import bdf.data.IBdfDatabase;
import bdf.util.DataHelpers;
public class BdfReader
{
protected BdfLookupTable lookupTable;
protected BdfObject bdf;
private void initNew() {
lookupTable = new BdfLookupTable(this);
bdf = new BdfObject(lookupTable);
}
public BdfReader() {
initNew();
}
public BdfReader(byte[] database) {
this(new BdfDatabase(database));
}
public BdfReader(IBdfDatabase database)
{
try
{
if(database.size() == 0) {
initNew();
return;
}
int upto = 0;
IBdfDatabase flag_ptr = database.getPointer(upto);
byte lookupTable_size_tag = BdfObject.getParentFlags(flag_ptr);
byte lookupTable_size_bytes = 0;
switch(lookupTable_size_tag)
{
case 0:
lookupTable_size_bytes = 4;
break;
case 1:
lookupTable_size_bytes = 2;
break;
case 2:
lookupTable_size_bytes = 1;
break;
}
if(lookupTable_size_bytes > database.size()) {
initNew();
return;
}
// Get the rest of the data
int bdf_size = BdfObject.getSize(flag_ptr);
if(bdf_size + lookupTable_size_bytes > database.size()) {
initNew();
return;
}
IBdfDatabase database_bdf = database.getPointer(upto, bdf_size);
upto += bdf_size;
// Get the lookup table
ByteBuffer lookupTable_size_buff = DataHelpers.getByteBuffer(database.getPointer(upto, lookupTable_size_bytes));
int lookupTable_size = 0;
switch(lookupTable_size_tag)
{
case 0:
lookupTable_size = lookupTable_size_buff.getInt();
break;
case 1:
lookupTable_size = 0xffff & lookupTable_size_buff.getShort();
break;
case 2:
lookupTable_size = 0xff & lookupTable_size_buff.get();
break;
}
if(lookupTable_size + lookupTable_size_bytes + bdf_size > database.size()) {
initNew();
return;
}
lookupTable = new BdfLookupTable(this, database.getPointer(lookupTable_size_bytes + upto, lookupTable_size));
bdf = new BdfObject(lookupTable, database_bdf);
}
catch(IndexOutOfBoundsException e) {
initNew();
}
}
public BdfDatabase serialize()
{
int[] locations = lookupTable.serializeGetLocations();
int bdf_size = bdf.serializeSeeker(locations);
int lookupTable_size = lookupTable.serializeSeeker(locations);
int lookupTable_size_bytes = 0;
byte lookupTable_size_tag = 0;
if(lookupTable_size > 65535) { // >= 2 ^ 16
lookupTable_size_tag = 0;
lookupTable_size_bytes = 4;
} else if(lookupTable_size > 255) { // >= 2 ^ 8
lookupTable_size_tag = 1;
lookupTable_size_bytes = 2;
} else { // < 2 ^ 8
lookupTable_size_tag = 2;
lookupTable_size_bytes = 1;
}
int upto = 0;
int database_size = bdf_size + lookupTable_size + lookupTable_size_bytes;
BdfDatabase database = new BdfDatabase(database_size);
bdf.serialize(database.getPointer(upto, bdf_size), locations, lookupTable_size_tag);
upto += bdf_size;
byte[] bytes = DataHelpers.serializeInt(lookupTable_size);
for(int i=0;i<lookupTable_size_bytes;i++) {
database.setByte(i + upto, bytes[i - lookupTable_size_bytes + 4]);
}
lookupTable.serialize(database.getPointer(upto + lookupTable_size_bytes, lookupTable_size), locations, (byte)0);
return database;
}
public BdfObject getObject() {
return bdf;
}
public BdfObject resetObject() {
bdf = new BdfObject(lookupTable);
return bdf;
}
public String serializeHumanReadable(BdfIndent indent)
{
ByteArrayOutputStream stream = new ByteArrayOutputStream();
try {
bdf.serializeHumanReadable(stream, indent, 0);
return stream.toString();
}
catch(IOException e) {
return "undefined";
}
}
public String serializeHumanReadable() {
return serializeHumanReadable(new BdfIndent("", ""));
}
public void serializeHumanReadable(OutputStream stream, BdfIndent indent) throws IOException
{
bdf.serializeHumanReadable(stream, indent, 0);
stream.write('\n');
stream.flush();
}
public void serializeHumanReadable(OutputStream stream) throws IOException {
serializeHumanReadable(stream, new BdfIndent("", ""));
}
}

View File

@ -0,0 +1,29 @@
package bdf.types;
import bdf.data.BdfStringPointer;
import bdf.util.BdfError;
public class BdfReaderHuman extends BdfReader
{
public BdfReaderHuman(String data)
{
BdfStringPointer ptr = new BdfStringPointer(data.toCharArray(), 0);
ptr.ignoreBlanks();
bdf = new BdfObject(lookupTable, ptr);
try {
ptr.ignoreBlanks();
}
catch(BdfError e) {
if(e.getType() == BdfError.ERROR_END_OF_FILE) {
return;
} else {
throw e;
}
}
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
}

View File

@ -2,15 +2,25 @@ package bdf.types;
public class BdfTypes public class BdfTypes
{ {
public static final byte BOOLEAN = 0; public static final byte UNDEFINED = 0;
public static final byte INTEGER = 1;
public static final byte LONG = 2; public static final byte BOOLEAN = 1;
public static final byte SHORT = 3; public static final byte INTEGER = 2;
public static final byte BYTE = 4; public static final byte LONG = 3;
public static final byte DOUBLE = 5; public static final byte SHORT = 4;
public static final byte FLOAT = 6; public static final byte BYTE = 5;
public static final byte STRING = 7; public static final byte DOUBLE = 6;
public static final byte ARRAY = 8; public static final byte FLOAT = 7;
public static final byte NAMED_LIST = 9;
public static final byte EMPTY = 10; public static final byte STRING = 8;
public static final byte ARRAY = 9;
public static final byte NAMED_LIST = 10;
public static final byte ARRAY_BOOLEAN = 11;
public static final byte ARRAY_INTEGER = 12;
public static final byte ARRAY_LONG = 13;
public static final byte ARRAY_SHORT = 14;
public static final byte ARRAY_BYTE = 15;
public static final byte ARRAY_DOUBLE = 16;
public static final byte ARRAY_FLOAT = 17;
} }

View File

@ -1,9 +1,15 @@
package bdf.types; package bdf.types;
import bdf.data.BdfDatabase; import java.io.IOException;
import java.io.OutputStream;
public interface IBdfType import bdf.data.IBdfDatabase;
interface IBdfType
{ {
public BdfDatabase serialize(); void getLocationUses(int[] locations);
public String serializeHumanReadable(); int serialize(IBdfDatabase database, int[] locations, byte flags);
int serializeSeeker(int[] locations);
void serializeHumanReadable(OutputStream stream, BdfIndent indent, int it) throws IOException;
} }

108
src/bdf/util/BdfError.java Normal file
View File

@ -0,0 +1,108 @@
package bdf.util;
import bdf.data.BdfStringPointer;
public class BdfError extends RuntimeException
{
private static final long serialVersionUID = -8731016151496842808L;
private static final String[] ERRORS = {
"Syntax error",
"End of file",
"Unescaped comment",
"Unescaped string",
"Number format error",
};
public static final int ERROR_SYNTAX = 0;
public static final int ERROR_END_OF_FILE = 1;
public static final int ERROR_UNESCAPED_COMMENT = 2;
public static final int ERROR_UNESCAPED_STRING = 3;
public static final int ERROR_NUMBER = 4;
public static BdfError createError(int errorID, BdfStringPointer ptr)
{
String error = ERRORS[errorID];
char[] array = ptr.getDataCharArray();
int location = ptr.getDataLocation();
int start_of_line = 0;
int line = 0;
int at = 0;
for(int i=0;i<location;i++)
{
if(array[i] == '\n') {
start_of_line = i + 1;
line += 1;
at = 0;
continue;
}
at += 1;
}
int line_size = 0;
String spacer = "";
for(int i=start_of_line;i<array.length;i++)
{
if(array[i] == '\n') {
break;
}
line_size += 1;
if(i == array.length - 1) {
break;
}
if(i < location)
{
if(array[i] == '\t') {
spacer += "\t";
continue;
}
spacer += " ";
}
}
char[] line_chars = new char[line_size];
for(int i=0;i<line_size;i++) {
line_chars[i] = array[start_of_line + i];
}
String message = "";
String error_short = error + " " + (line + 1) + ":" + (at + 1);
message += error_short + "\n";
message += new String(line_chars) + "\n";
message += spacer;
message += "^";
BdfError bdf_error = new BdfError(message);
bdf_error.error_short = error_short;
bdf_error.type = errorID;
return bdf_error;
}
private String error_short;
private int type;
public int getType() {
return type;
}
public String getErrorShort() {
return error_short;
}
private BdfError(String message) {
super(message);
}
}

View File

@ -3,10 +3,15 @@ package bdf.util;
import java.nio.ByteBuffer; import java.nio.ByteBuffer;
import bdf.data.BdfDatabase; import bdf.data.BdfDatabase;
import bdf.data.IBdfDatabase;
public class DataHelpers public class DataHelpers
{ {
public static ByteBuffer getByteBuffer(BdfDatabase db) { private static final char[] HEX = {
'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'
};
public static ByteBuffer getByteBuffer(IBdfDatabase db) {
return ByteBuffer.wrap(db.getBytes()); return ByteBuffer.wrap(db.getBytes());
} }
@ -14,11 +19,11 @@ public class DataHelpers
return new BdfDatabase(buffer.array()); return new BdfDatabase(buffer.array());
} }
public static BdfDatabase serializeInt(int value) public static byte[] serializeInt(int value)
{ {
ByteBuffer buffer = ByteBuffer.allocate(Integer.SIZE/8); ByteBuffer buffer = ByteBuffer.allocate(4);
buffer.putInt(value); buffer.putInt(value);
return getDatabase(buffer); return buffer.array();
} }
public static boolean bytesAreEqual(byte[] b1, byte[] b2) public static boolean bytesAreEqual(byte[] b1, byte[] b2)
@ -37,17 +42,16 @@ public class DataHelpers
return true; return true;
} }
public static String replaceInString(String string, byte find, String replace) public static String replaceInString(String string, char find, String replace)
{ {
// Convert the string to bytes // Convert the string to bytes
byte[] string_b = string.getBytes();
String string_modified = new String(); String string_modified = new String();
// Loop over the string // Loop over the string
for(int i=0;i<string_b.length;i++) for(int i=0;i<string.length();i++)
{ {
// Is the byte to find the byte at this part of the string // Is the byte to find the byte at this part of the string
if(find == string_b[i]) if(find == string.charAt(i))
{ {
// Add the data to replace to the string // Add the data to replace to the string
string_modified += replace; string_modified += replace;
@ -64,22 +68,110 @@ public class DataHelpers
return string_modified; return string_modified;
} }
public static String replaceInString(String string, char find, String replace) { public static char[] replaceInCharArray(char[] chars, char find, String replace)
return replaceInString(string, (byte)find, replace); {
char[] replace_chars = replace.toCharArray();
int replace_size = replace.length();
int size = 0;
// Get the new size of the char array
for(int i=0;i<chars.length;i++) {
if(chars[i] == find) {
size += replace_size;
} else {
size += 1;
}
}
char[] chars_modified = new char[size];
int upto = 0;
// Replace the contents of the char array
for(int i=0;i<chars.length;i++)
{
if(chars[i] == find)
{
for(int j=0;j<replace_size;j++) {
chars_modified[upto+j] = replace_chars[j];
}
upto += replace_size;
}
else
{
chars_modified[upto] = chars[i];
upto += 1;
}
}
return chars_modified;
} }
public static String serializeString(String string) public static String serializeString(String string)
{ {
// Serialize the string char[] string_chars = string.toCharArray();
String serialized = string;
// Replace some parts of the string // Replace some parts of the string
serialized = replaceInString(serialized, '"', "\\\""); string_chars = replaceInCharArray(string_chars, '\\', "\\\\");
string_chars = replaceInCharArray(string_chars, '"', "\\\"");
string_chars = replaceInCharArray(string_chars, '\n', "\\n");
string_chars = replaceInCharArray(string_chars, '\t', "\\t");
// Replace all the unreadable parts of the string
{
int size = 0;
for(int i=0;i<string_chars.length;i++)
{
char c = string_chars[i];
if(c < 0x20 || (c > 0x7e && c < 0xa1) || c == 0xad)
{
// Will be in the format \u0000
size += 6;
}
else
{
size += 1;
}
}
char[] chars = new char[size];
int upto = 0;
for(int i=0;i<string_chars.length;i++)
{
char c = string_chars[i];
if(c < 0x20 || (c > 0x7e && c < 0xa1) || c == 0xad)
{
// Will be in the format \u0000
chars[upto] = '\\';
chars[upto+1] = 'u';
chars[upto+2] = HEX[(c & 0xf000) >> 12];
chars[upto+3] = HEX[(c & 0x0f00) >> 8];
chars[upto+4] = HEX[(c & 0x00f0) >> 4];
chars[upto+5] = HEX[(c & 0x000f)];
upto += 6;
}
else
{
chars[upto] = string_chars[i];
upto += 1;
}
}
string_chars = chars;
}
// Add quotes to the string // Add quotes to the string
serialized = "\"" + serialized + "\""; string = "\"" + new String(string_chars) + "\"";
// Return the serialized string // Return the serialized string
return serialized; return string;
} }
} }

View File

@ -1,59 +0,0 @@
package bdf.util;
import java.io.FileInputStream;
import java.io.IOException;
public class FileHelpers
{
public static byte[] readAll(FileInputStream in)
{
try
{
// Get bytes to return
int available = in.available();
byte[] bytes = new byte[available];
// Loop over the available bytes
for(int i=0;i<available;i++)
{
// Add the next byte from the stream to the bytes
bytes[i] = (byte) in.read();
}
// Send the bytes collected from the file stream back
return bytes;
}
catch (IOException e)
{
// Throw the IOException as a runtime exception
throw new RuntimeException(e);
}
}
public static byte[] readAll(String path) throws IOException
{
// Create the file input stream
FileInputStream in = new FileInputStream(path);
// Load all of its data
byte[] data = readAll(in);
// Close the file input stream
in.close();
// Send back the data
return data;
}
public static byte[] readAllIgnoreErrors(String path)
{
try {
return readAll(path);
}
catch(IOException e) {
return new byte[0];
}
}
}

36
src/tests/Tests.java Executable file
View File

@ -0,0 +1,36 @@
package tests;
import java.io.FileInputStream;
import java.io.IOException;
import bdf.data.IBdfDatabase;
import bdf.types.BdfReader;
import bdf.types.BdfReaderHuman;
public class Tests
{
public static void displayHex(IBdfDatabase db)
{
char[] hex_chars = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'};
System.out.print("Size: " + db.size() + ", Hex: ");
for(int i=0;i<db.size();i++)
{
int b = db.getByte(i);
if(b < 0) b += 128;
System.out.print(hex_chars[b / 16]);
System.out.print(hex_chars[b % 16]);
System.out.print(' ');
}
System.out.println();
}
public static void main(String[] args) throws IOException
{
new BdfReaderHuman("int(NaND)");
}
}