Compare commits

...

33 Commits
v1.0 ... master

Author SHA1 Message Date
josua a75f208b02 Fixed an issue with the README, improved error messages 2020-09-13 16:17:53 +10:00
Josua Robson 87d916b0b4
Merge pull request from jsrobson10/dev
Merge dev into master
2020-09-09 14:06:35 +10:00
jsrobson10 960d0c3b17 Updated the README, added support for non-real integers, fixed issues
when trying to parse garbage data, cleaned up some code.
2020-09-09 14:02:38 +10:00
jsrobson10 04657bc6ad Updated gitignore 2020-08-19 13:40:15 +10:00
josua 01348a5adf Got something working 2020-08-18 22:52:08 +10:00
josua 89426bff98 Added human readable de-serialization, modified the human readable
layout slightly to allow for empty primitive arrays, added error
handling for human readable data, started working on README
2020-08-17 22:54:22 +10:00
josua 8ba902bacd Trying to compress the format as much as possible 2020-08-14 12:25:04 +10:00
josua f73eb0454f Got a functional version of BDF 3 2020-07-27 11:11:01 +10:00
jsrobson10 31e6b9b637 Added emoji support 2020-07-24 17:54:56 +10:00
josua b62c43e892 Started working on a lookup table system for BdfNamedList to save memory and space 2020-07-24 11:26:52 +10:00
jsrobson10 42374c79c5
Added implementation details to the README 2020-07-22 15:48:08 +10:00
jsrobson10 471fbfe42b
Update README.md 2020-07-22 15:27:04 +10:00
jsrobson10 83778d8379
Update README.md 2020-07-22 15:25:49 +10:00
jsrobson10 7a50410132 Moved the compressed variable set from a static method to the constructor 2020-07-22 11:34:03 +10:00
josua 1c803c6125 Forgot to store compression flag to BdfFileManager 2020-07-22 11:30:45 +10:00
josua 984c78020a Added the compressed file manager functionality to the BdfFileManager
class and fixed issues with boolean.
2020-07-21 21:17:51 +10:00
jsrobson10 23a49277c1 Modified the human readable serialization to use streams instead of
strings, to make human readable formats able to handle an output of
greater than 2GB.
2020-06-20 13:38:59 +10:00
josua 39de6a5df4 Improved efficiency of the bdf format by implementing a "pointer"
solution. Fixed the issue with very large files taking up too much
memory. Made the BdfDatabase object more useful in general.
2020-06-18 22:54:27 +10:00
jsrobson10 03e4b583f4 Fixed some issues with indenting 2020-03-24 10:11:32 +11:00
jsrobson10 47e7822177 Added indentation support to the human readable serialisation 2020-03-24 09:39:56 +11:00
jsrobson10 a05cbe7375 Added support for compression 2020-03-19 14:17:55 +11:00
josua 376e5c5261 Added arrays for all the primitives 2020-02-27 13:01:04 +11:00
josua 8ed682653e Added the ByteBuffer type 2019-09-21 13:27:56 +10:00
josua 5a471d0f8c Fixed some issues with iteration, updated readme 2019-07-26 20:07:05 +10:00
josua bfafa4658e Cleaned the code, updated README, made the structure more forgiving 2019-07-26 12:25:16 +10:00
jsrobson10 3b94ac2646
Merge pull request from jsrobson10/save
Save
2019-07-10 14:42:09 +10:00
jsrobson10 7fd421d2b8 Added some more information to the README 2019-07-10 12:57:01 +10:00
jsrobson10 5d6285435a Added information to the README about saving classes 2019-07-10 12:54:10 +10:00
jsrobson10 269ab3a386 Added lots of helper functions to BdfNamedList and BdfObject 2019-07-10 12:37:51 +10:00
jsrobson10 8ca912589b Added lots of helper functions to BdfObject and BdfNamedList 2019-07-10 12:37:21 +10:00
jsrobson10 da3b6a117d Changed BdfNamedList in IBdfClassManager to BdfObject 2019-07-10 12:15:25 +10:00
jsrobson10 ad7d8cb04a Made some modifications to the README 2019-07-10 12:02:36 +10:00
jsrobson10 f01696328b Added some functionality to allow saving classes easier 2019-07-10 12:00:39 +10:00
25 changed files with 3055 additions and 609 deletions

2
.gitignore vendored
View File

@ -1 +1,3 @@
/bin/
/*.bdf
/*.hbdf

332
README.md
View File

@ -3,23 +3,28 @@
### Links
- <a href="#overview">Overview</a>
- <a href="#languages">Languages</a>
- <a href="#data-types">Data types</a>
- <a href="#creating-an-object">Creating an object</a>
- <a href="#arrays">Arrays</a>
- <a href="#named-lists">Named lists</a>
- <a href="#example-bdf-program">Example BDF program</a>
- <a href="#human-readable-representation">Human readable representation</a>
- <a href="#special-notes">Special notes</a>
### Overview
Binary Data Format (or BDF) is designed to store data in a tag-like binary structure,
like Notch's NBT format, but also open source and free like JSON. The format is
fast and allows multiple data types, it uses 32-bit integers, so BDF files can
be fast and work well on 32-bit systems, but have a maximum size of 2 GB.
BDF allows human readable serialization to see what is going on for debugging
purposes, but it currently can't parse the result to an object.
Binary Data Format (BDF) is a statically typed data representation
format. It was made to be free, fast, compact, and seamlessly
convertable between its human readable and binary representations.
### Languages
- <a href="https://github.com/jsrobson10/BdfCpp">C++</a>
- Java
### Data types
- Undefined
- Boolean
- Integer
- Long
@ -27,20 +32,33 @@ purposes, but it currently can't parse the result to an object.
- Byte
- Double
- Float
- Boolean Array
- Integer Array
- Long Array
- Short Array
- Byte Array
- Double Array
- Float Array
- String
- Array
- Named List
- Empty
### Creating an object
You will need to create a new object to store any data, use a `BdfObject` instance.
You can input serialized data into `BdfObject` via a `BdfDatabase`.
You will need to generate a BdfObject to serialize anything,
this can be done by first generating a BdfReader, or generating
a new object via an existing BdfObject.
```java
// New BDF object
BdfObject bdf = new BdfObject();
// Create a reader object
BdfReader reader = new BdfReader();
// Get the BdfObject instance
BdfObject bdf = reader.getObject();
// Generate another BdfObject instance
BdfObject bdf_new = bdf.newObject();
// Get an integer
int v = bdf.getInteger();
@ -48,6 +66,16 @@ int v = bdf.getInteger();
// Set an integer
bdf.setInteger(5);
// Set an integer with an automatic type
bdf.setAutoInt(53);
// Set a primitive array of ints
int intArray[] = {3, 4, 5, 6};
bdf.setIntegerArray(intArray);
// Get a byte array
byte[] byteArray = bdf.getByteArray();
// Get the type of variable of the object
int type = bdf.getType();
@ -58,178 +86,216 @@ if(type == BdfTypes.INTEGER)
}
// Serialize the BDF object
byte[] data = bdf.serialize().getBytes();
byte[] data = bdf->serialize(&data, &data_size);
// Load another reader object from the serialized bytes
BdfReader reader2 = new BdfReader(data);
/*
A reader object can be serialized to the human readable
representation as a string or sent over a stream
*/
reader2.serializeHumanReadable(System.out, new BdfIndent("\t", "\n"));
String data_hr = reader2.serializeHumanReadable(new BdfIndent("\t", "\n"));
// Load another BDF object with the serialized bytes
BdfObject bdf2 = new BdfObject(new BdfDatabase(data));
```
A `BdfFileManager`
instance can be used in the same way as a `BdfObject`, but it also needs a String parameter
for the path of the file. The file can be written with `BdfFileManager.saveDatabase()`.
A `BdfFileManager` is an instance of `BdfObject`, a `BdfFileManager` can be casted to
a `BdfObject`.
```java
// Open a file
BdfFileManager bdf = new BdfFileManager("file.bdf");
// Save the database
bdf.saveDatabase();
// The file can be casted to a BdfObject
BdfObject bdf2 = (BdfObject) bdf;
// Bdf
System.out.println(bdf instanceof BdfObject); // true
// A reader object can be loaded from a human readable object
BdfReader reader3 = new BdfReaderHuman(data_hr);
```
### Arrays
Arrays can be used to store lists of information, they hold `BdfObject`.
The array is called with `new BdfArray()`. It can hold information, get
the size of the array with `BdfArray.size()`, remove elements with
`BdfArray.remove(index)`, set indexes with `BdfArray.set(index, BdfObject)`,
and add elements with `BdfArray.add(BdfObject)`. Arrays also
have support for Iterators and are an instance of `Iterable`.
Arrays can be used to store chunks of information, they hold instances of
BdfObject. Arrays can also be iterated over just like any other array.
```java
// New BDF Object
BdfObject bdf = new BdfObject();
BdfReader reader = new BdfReader();
BdfObject bdf = reader.getObject();
// New BDF Array
BdfArray array = new BdfArray();
// Can be created from a bdf object
BdfArray array = bdf.newArray();
// Size
// Get the length of an array
int size = array.size();
// Remove
array.remove(3); // Could be any number
// Remove any index from an array
array.remove(3);
// Set - Could be any number with any object
array.set(4, BdfObject.with("A String"));
// Set an object to an index of an array
array.set(4, bdf.newObject().setString("A String"));
// Add - Could be any object
array.add(BdfObject.with(53));
// Add an object to an array
array.add(bdf.newObject().setByte((byte)53));
// Set the array to the bdf object
bdf.setArray(array);
// Iterate over an array
for(BdfObject o : array)
{
}
```
### Named lists
Named lists can be used to store data under strings,
Named lists can be used to store data under ids/strings
to be used like variables in a program. A named list
can be created with `new BdfNamedList()` and it
has the ability to set with `BdfNamedList.set(String, BdfObject)`,
remove with `BdfNamedList.remove(String)`, and check
for a key with `BdfNamedList.contains(String)`. It also has
features to get all the keys with `BdfNamedList.getKeys()`.
Named lists also have Iterator support and are an instance of
`Iterable`.
can be created similar to an array.
```java
// New bdf named list
BdfNamedList list = new BdfNamedList();
BdfReader reader = new BdfReader();
BdfObject bdf = reader.getObject();
// Set an element with a value
list.set("key1", BdfObject.with(5));
// New named list
BdfNamedList list = bdf.newNamedList();
// Set an element to the named list
list.set("key1", bdf.newObject().setInteger(5));
// Use ids instead of strings for optimisation
// if set/get is being called multiple times
// on the same key.
int key2 = bdf.getKeyLocation("key2");
list.set(key2, bdf.newObject().setFloat(42.0F));
// Get an elements value
int v = list.get("key1").getInteger();
// Check if an element exists
boolean has_key = list.contains("key1");
bool has_key = list.contains("key1");
// Get the lists keys
String[] keys = list.getKeys();
int[] keys = list.getKeys();
// Iterate over the lists keys
for(String key : list)
for(int key : keys)
{
// Get the keys name
String key_name = bdf.getKeyName(key);
}
```
### Human readable representation
### Example BDF program
A big part of binary data format is the human readable
representation. It has a JSON-like syntax.
This can be used with config files and to modify/view
binaries. A big advantage to using the human readable
representation in configuration files is its support
for comments.
```hbdf
/*
A Named List is represented
by an opening tag and a closing
tag { }
*/
{
/*
A key value pair can be stored
within a Named List with a string
property
*/
"hello": "world",
/*
Integers can be stored here too.
They have a character at the end
to say what type they are.
The tag at the end can be:
- I: Integer - a value between -2^31 and 2^31 - 1
- S: Short - a value between -32768 and 32767
- L: Long - a value between -2^63 and 2^63 - 1
- B: Byte - a value between -128 and 127
- D: Double - has 15 decimal digits of precision
- F: Float - has 7 decimal digits of precision
*/
"number": 42I,
"byte": -23B,
"decimal": 73.5D,
/*
This is a boolean. It can
be true or false.
*/
"boolTest": false,
/*
Primitive arrays are represented
by a type, an opening tag, and a
closing tag. They are like an array
but they contain only 1 data type.
The tag at the start can be:
- int
- short
- long
- byte
- double
- float
- bool
*/
"intArray": int (
64I, 42I, 63I,
22I, 96I, -12I,
),
/*
The double and float types support
Infinity, -Infinity, and NaN.
They also support both really
high and really low value numbers.
*/
"doubleArray": double (
42.5D, -20D, 400D,
NaND, -InfinityD, InfinityD,
5.3e-200D, 4e+120D, 2.2e200D,
),
/*
Arrays are enclosed by an opening
tag and a closing tag [ ]
Like the Named List, it can hold
any data type.
*/
"people": [
{"name": "foo", "age": 60B},
{"name": "bar", "age": 21B},
],
// This is a single-line comment
/* This is a multi-line comment */
}
```
### Special notes
Don't mix bdf types between different
readers, this will cause problems.
```java
// Create a new BdfObject instance
BdfObject bdf = new BdfObject();
BdfReader reader1 = new BdfReader();
BdfReader reader2 = new BdfReader();
// Create a named list
BdfNamedList bdf_nl = new BdfNamedList();
BdfObject bdf1 = reader1.getObject();
BdfObject bdf2 = reader2.getObject();
// Add some variables to the named list
bdf_nl.set("boolean", BdfObject.with(true));
bdf_nl.set("an_int", BdfObject.with((int)53));
bdf_nl.set("double", new BdfObject().setDouble(632.5));
// Don't do this
bdf1.setNamedList(bdf2.newNamedList());
// Output some checks on BdfNamedList
System.out.println(bdf_nl.contains("an_int")); // true
System.out.println(bdf_nl.contains("this_dosn't_exist")); // false
// Or this
bdf1.setArray(bdf2.newArray());
// Create an array
BdfArray bdf_array = new BdfArray();
// Or this
BdfNamedList nl = bdf1.newArray();
nl.set("illegal", bdf2.newObject().setString("action"));
// Add some values to the array
bdf_array.add(BdfObject.with("Hello, World!"));
bdf_array.add(BdfObject.with(1234567890L));
bdf_array.set(1, BdfObject.with((short)432));
```
// Output the size of the array
System.out.println(bdf_array.size()); // 2
// Output the type of the 2nd item in the array, value types are in BdfTypes
System.out.println(bdf_array.get(1).getType()); // 3 (BdfTypes.SHORT)
// Save the array to the named list
bdf_nl.set("array", BdfObject.with(bdf_array));
// Set the named list to the bdf object
bdf.setNamedList(bdf_nl);
// Serialize the data
byte[] bdf_data = bdf.serialize().getBytes();
// Load the serialized data
BdfObject bdf2 = new BdfObject(new BdfDatabase(bdf_data));
// Show the human readable serialized data
System.out.println(bdf2.serializeHumanReadable()); // {"boolean": true, "an_int": 53I, "double": 632.5D, "array": ["Hello, World!", 432S]}
// Show the value of the boolean in the named list
System.out.println(bdf2.getNamedList().get("boolean").getBoolean()); // true
// Show the value of item 0 in the array
System.out.println(bdf2.getNamedList().get("array").getArray().get(0).getString()); // Hello, World!
// Check if the double exists
System.out.println(bdf2.getNamedList().contains("double")); // true
// Remove the double from the named list
bdf2.getNamedList().remove("double");
// Check if the double exists
System.out.println(bdf2.getNamedList().contains("double")); // false
```

Binary file not shown.

View File

@ -1,75 +0,0 @@
package bdf;
import bdf.data.BdfDatabase;
import bdf.types.BdfArray;
import bdf.types.BdfNamedList;
import bdf.types.BdfObject;
public class Tests {
public static void main(String[] args)
{
// Create a new BdfObject instance
BdfObject bdf = new BdfObject();
// Create a named list
BdfNamedList bdf_nl = new BdfNamedList();
// Add some variables to the named list
bdf_nl.set("boolean", BdfObject.with(true));
bdf_nl.set("an_int", BdfObject.with((int)53));
bdf_nl.set("double", new BdfObject().setDouble(632.5));
// Output some checks on BdfNamedList
System.out.println(bdf_nl.contains("an_int")); // true
System.out.println(bdf_nl.contains("this_dosn't_exist")); // false
// Create an array
BdfArray bdf_array = new BdfArray();
// Add some values to the array
bdf_array.add(BdfObject.with("Hello, World!"));
bdf_array.add(BdfObject.with(1234567890L));
bdf_array.set(1, BdfObject.with((short)432));
// Output the size of the array
System.out.println(bdf_array.size()); // 2
// Output the type of the 2nd item in the array, value types are in BdfTypes
System.out.println(bdf_array.get(1).getType()); // 3 (BdfTypes.SHORT)
// Save the array to the named list
bdf_nl.set("array", BdfObject.with(bdf_array));
// Set the named list to the bdf object
bdf.setNamedList(bdf_nl);
// Serialize the data
byte[] bdf_data = bdf.serialize().getBytes();
// Load the serialized data
BdfObject bdf2 = new BdfObject(new BdfDatabase(bdf_data));
// Show the human readable serialized data
System.out.println(bdf2.serializeHumanReadable()); // {"boolean": true, "an_int": 53I, "double": 632.5D, "array": ["Hello, World!", 432S]}
// Show the value of the boolean in the named list
System.out.println(bdf2.getNamedList().get("boolean").getBoolean()); // true
// Show the value of item 0 in the array
System.out.println(bdf2.getNamedList().get("array").getArray().get(0).getString()); // Hello, World!
// Check if the double exists
System.out.println(bdf2.getNamedList().contains("double")); // true
// Remove the double from the named list
bdf2.getNamedList().remove("double");
// Check if the double exists
System.out.println(bdf2.getNamedList().contains("double")); // false
}
}

View File

@ -1,62 +1,170 @@
package bdf.data;
import java.nio.charset.StandardCharsets;
import java.io.IOException;
import java.io.OutputStream;
public class BdfDatabase
public class BdfDatabase implements IBdfDatabase
{
protected byte[] database = null;
static final int STREAM_CHUNK_SIZE = 1024*1024;
public BdfDatabase() {
this.database = new byte[0];
byte[] database;
public BdfDatabase(byte[] bytes) {
database = bytes;
}
public BdfDatabase(String database) {
this.database = database.getBytes();
public BdfDatabase(String str) {
this(str.getBytes());
}
public BdfDatabase(byte ...database) {
this.database = database;
public BdfDatabase(int size) {
this.database = new byte[size];
}
public BdfDatabase getAt(int start, int end)
@Override
public IBdfDatabase getCopy(int start, int end)
{
byte[] split = new byte[end - start];
byte[] database = new byte[end - start];
for(int i=start;i<end;i++) {
split[i-start] = this.database[i];
for(int i=0;i<end-start;i++) {
database[i] = this.database[i + start];
}
return new BdfDatabase(split);
return new BdfDatabase(database);
}
public int length() {
return this.database.length;
}
public byte[] getBytes() {
return this.database;
@Override
public byte getByte() {
return database[0];
}
@Override
public byte getByte(int i) {
return this.database[i];
return database[i];
}
public String getString() {
return new String(this.database, StandardCharsets.UTF_8);
}
public static BdfDatabase add(BdfDatabase d1, BdfDatabase d2)
@Override
public byte[] getBytes(int start, int size)
{
byte[] added = new byte[d1.length() + d2.length()];
byte[] database = new byte[size];
for(int i=0;i<d1.length();i++) {
added[i] = d1.getByte(i);
for(int i=0;i<size;i++) {
database[i] = this.database[i + start];
}
for(int i=0;i<d2.length();i++) {
added[d1.length()+i] = d2.getByte(i);
return database;
}
@Override
public byte[] getBytes() {
return getBytes(0, database.length);
}
@Override
public IBdfDatabase getPointer(int location, int size)
{
BdfDatabasePointer db = new BdfDatabasePointer();
db.database = database;
db.location = location;
db.size = size;
return db;
}
@Override
public IBdfDatabase getPointer(int location) {
return getPointer(location, location);
}
@Override
public int size() {
return database.length;
}
@Override
public String getString() {
return new String(database);
}
@Override
public void writeToStream(OutputStream stream, int start, int size) throws IOException
{
for(int i=0;i<size;i+=STREAM_CHUNK_SIZE)
{
if(size - i < STREAM_CHUNK_SIZE) {
stream.write(getBytes(i + start, size - i));
} else {
stream.write(getBytes(i + start, STREAM_CHUNK_SIZE));
}
}
}
@Override
public void writeToStream(OutputStream stream) throws IOException {
writeToStream(stream, 0, database.length);
}
public static IBdfDatabase add(IBdfDatabase b1, IBdfDatabase b2)
{
byte[] bytes = new byte[b1.size() + b2.size()];
int b1_size = b1.size();
for(int i=0;i<bytes.length;i++)
{
if(i >= b1_size) {
bytes[i] = b2.getByte(i - b1_size);
} else {
bytes[i] = b1.getByte(i);
}
}
return new BdfDatabase(added);
return new BdfDatabase(bytes);
}
@Override
public void setByte(int pos, byte b) {
database[pos] = b;
}
@Override
public void setByte(byte b) {
database[0] = b;
}
@Override
public void setBytes(IBdfDatabase bytes) {
setBytes(bytes, 0, bytes.size());
}
@Override
public void setBytes(IBdfDatabase bytes, int offset) {
setBytes(bytes, offset, bytes.size());
}
@Override
public void setBytes(IBdfDatabase bytes, int offset, int length)
{
for(int i=0;i<length;i++) {
database[offset + i] = bytes.getByte(i);
}
}
@Override
public void setBytes(byte[] bytes, int offset, int length)
{
for(int i=0;i<length;i++) {
database[offset + i] = bytes[i];
}
}
@Override
public void setBytes(byte[] bytes, int offset) {
setBytes(bytes, offset, bytes.length);
}
@Override
public void setBytes(byte[] bytes) {
setBytes(bytes, 0, bytes.length);
}
}

View File

@ -0,0 +1,160 @@
package bdf.data;
import java.io.IOException;
import java.io.OutputStream;
public class BdfDatabasePointer implements IBdfDatabase
{
byte[] database;
int location;
int size;
public BdfDatabasePointer(byte[] bytes) {
database = bytes;
size = bytes.length;
location = 0;
}
public BdfDatabasePointer(String str) {
this(str.getBytes());
}
public BdfDatabasePointer(int size) {
this.database = new byte[size];
this.location = 0;
this.size = size;
}
BdfDatabasePointer() {
}
@Override
public IBdfDatabase getCopy(int start, int end)
{
byte[] database = new byte[end - start];
for(int i=0;i<end-start;i++) {
database[i] = this.database[i + start + location];
}
return new BdfDatabase(database);
}
@Override
public byte getByte() {
return database[location];
}
@Override
public byte getByte(int i) {
return database[location + i];
}
@Override
public byte[] getBytes(int start, int size)
{
byte[] database = new byte[size];
for(int i=0;i<size;i++) {
database[i] = this.database[i + location + start];
}
return database;
}
@Override
public byte[] getBytes() {
return getBytes(0, size);
}
@Override
public IBdfDatabase getPointer(int location, int size)
{
BdfDatabasePointer db = new BdfDatabasePointer();
db.database = database;
db.location = this.location + location;
db.size = size;
return db;
}
@Override
public IBdfDatabase getPointer(int location) {
return getPointer(location, size - location);
}
@Override
public int size() {
return size;
}
@Override
public String getString() {
return new String(database, location, size);
}
@Override
public void writeToStream(OutputStream stream, int start, int size) throws IOException
{
for(int i=0;i<size;i+=BdfDatabase.STREAM_CHUNK_SIZE)
{
if(size - i < BdfDatabase.STREAM_CHUNK_SIZE) {
stream.write(getBytes(i + start, size - i));
} else {
stream.write(getBytes(i + start, BdfDatabase.STREAM_CHUNK_SIZE));
}
}
}
@Override
public void writeToStream(OutputStream stream) throws IOException {
writeToStream(stream, 0, size);
}
@Override
public void setByte(int pos, byte b) {
database[pos + location] = b;
}
@Override
public void setByte(byte b) {
database[location] = b;
}
@Override
public void setBytes(IBdfDatabase bytes) {
setBytes(bytes, 0, bytes.size());
}
@Override
public void setBytes(IBdfDatabase bytes, int offset) {
setBytes(bytes, offset, bytes.size());
}
@Override
public void setBytes(IBdfDatabase bytes, int offset, int length)
{
for(int i=0;i<length;i++) {
database[offset + location + i] = bytes.getByte(i);
}
}
@Override
public void setBytes(byte[] bytes, int offset, int length)
{
for(int i=0;i<length;i++) {
database[offset + location + i] = bytes[i];
}
}
@Override
public void setBytes(byte[] bytes, int offset) {
setBytes(bytes, offset, bytes.length);
}
@Override
public void setBytes(byte[] bytes) {
setBytes(bytes, 0, bytes.length);
}
}

View File

@ -0,0 +1,290 @@
package bdf.data;
import bdf.util.BdfError;
public class BdfStringPointer
{
char[] data;
int offset;
public BdfStringPointer(char[] data, int offset) {
this.data = data;
this.offset = offset;
}
public BdfStringPointer getPointer(int offset) {
return new BdfStringPointer(data, this.offset + offset);
}
public void increment(int amount) {
offset += amount;
}
public void increment() {
offset += 1;
}
public char[] getDataCharArray() {
return data;
}
public int getDataLocation() {
return offset;
}
public int getDataLength() {
return data.length;
}
public char[] getCharArray(int offset, int length)
{
char[] array = new char[length];
for(int i=0;i<length;i++) {
array[i] = data[i + offset + this.offset];
}
return array;
}
public char[] getCharArray(int length) {
return getCharArray(0, length);
}
public char getChar(int i) {
return data[offset + i];
}
public char getChar() {
return data[offset];
}
public void ignoreBlanks()
{
for(;;)
{
if(offset >= data.length) {
throw BdfError.createError(BdfError.ERROR_END_OF_FILE, this);
}
char c = getChar();
// Comments
if(c == '/' && offset < data.length)
{
char c2 = getChar(1);
// Line comment
if(c2 == '/')
{
for(;;)
{
if(offset + 1 >= data.length) {
break;
}
increment();
c = getChar();
if(c == '\n') {
break;
}
}
}
// Multi-line comment
else if(c2 == '*')
{
for(;;)
{
if(offset + 1 >= data.length) {
throw BdfError.createError(BdfError.ERROR_UNESCAPED_COMMENT, getPointer(-1));
}
increment();
c = getChar();
if(c == '*' && offset < data.length && getChar(1) == '/') {
increment();
break;
}
}
}
else {
return;
}
}
else if(!(c == '\n' || c == '\t' || c == ' ')) {
return;
}
increment();
}
}
// In the format "abc\n\t\u0003..."
public String getQuotedString()
{
if(getChar() != '"') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, this);
}
increment();
String str = "";
for(;;)
{
if(offset >= data.length) {
throw BdfError.createError(BdfError.ERROR_UNESCAPED_STRING, this);
}
char c = getChar();
// Check for back slashes
if(c == '\\')
{
increment();
c = getChar();
switch(c)
{
case 'n':
str += "\n";
increment();
break;
case 't':
str += "\t";
increment();
break;
case '"':
str += "\"";
increment();
break;
case '\\':
str += "\\";
increment();
break;
case '\n':
str += "\n";
increment();
break;
case 'u': // \u0000
{
if(offset + 5 >= data.length) {
throw BdfError.createError(BdfError.ERROR_UNESCAPED_STRING, getPointer(1));
}
char[] hex = getCharArray(1, 4);
char unicode = (char)0;
int m = 1;
for(int j=hex.length-1;j>=0;j--)
{
c = hex[j];
if(c >= '0' && c <= '9') {
unicode += (char)(m * (c - '0'));
}
else if(c >= 'a' && c <= 'f') {
unicode += (char)(m * (c - 'a' + 10));
}
else {
throw BdfError.createError(BdfError.ERROR_SYNTAX, getPointer(1 + (hex.length-j-1)));
}
m *= 16;
}
str += unicode;
increment(5);
break;
}
default:
str += "\\" + c;
}
}
else if(c == '\n') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, this);
}
else if(c == '"') {
increment();
break;
}
else {
increment();
str += c;
}
}
return str;
}
public boolean isNext(String check)
{
if(check.length() + offset > data.length) {
return false;
}
for(int i=0;i<check.length();i++)
{
char c = getChar(i);
c = (char)((c >= 'A' && c <= 'Z') ? (c + 32) : c);
if(c != check.charAt(i)) {
return false;
}
}
increment(check.length());
return true;
}
public boolean isInteger()
{
for(int i=offset;i<data.length;i++)
{
char c = data[i];
switch(c)
{
case 'I':
return true;
case 'L':
return true;
case 'S':
return true;
case 'B':
return true;
case 'D':
return false;
case 'F':
return false;
case 'e':
continue;
case 'E':
continue;
case '.':
continue;
case '-':
continue;
}
if(c >= '0' && c <= '9') {
continue;
}
throw BdfError.createError(BdfError.ERROR_SYNTAX, new BdfStringPointer(data, i));
}
throw BdfError.createError(BdfError.ERROR_END_OF_FILE, new BdfStringPointer(data, data.length - 1));
}
}

View File

@ -0,0 +1,34 @@
package bdf.data;
import java.io.IOException;
import java.io.OutputStream;
public interface IBdfDatabase
{
public IBdfDatabase getCopy(int start, int end);
public IBdfDatabase getPointer(int location, int size);
public IBdfDatabase getPointer(int location);
public void writeToStream(OutputStream stream, int start, int size) throws IOException;
public void writeToStream(OutputStream stream) throws IOException;
public int size();
public byte[] getBytes();
public byte[] getBytes(int start, int size);
public byte getByte();
public byte getByte(int i);
public String getString();
public void setBytes(byte[] bytes, int offset, int length);
public void setBytes(byte[] bytes, int offset);
public void setBytes(byte[] bytes);
public void setBytes(IBdfDatabase bytes, int offset, int length);
public void setBytes(IBdfDatabase bytes, int offset);
public void setBytes(IBdfDatabase bytes);
public void setByte(int pos, byte b);
public void setByte(byte b);
}

View File

@ -1 +0,0 @@
/UndefinedKeyException.class

View File

@ -1,11 +0,0 @@
package bdf.exception;
public class UndefinedKeyException extends RuntimeException
{
private static final long serialVersionUID = 1L;
public UndefinedKeyException(String key)
{
super("The key \""+key+"\" has not been defined.");
}
}

View File

@ -1 +0,0 @@
/BdfFileManager.class

View File

@ -1,64 +0,0 @@
package bdf.file;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import bdf.data.BdfDatabase;
import bdf.types.BdfObject;
import bdf.util.FileHelpers;
public class BdfFileManager extends BdfObject
{
protected String path;
private static BdfDatabase init(String path)
{
// Get the file
File file = new File(path);
// Does the file have read access
if(file.canRead())
{
// Return the files contents as a database
return new BdfDatabase(FileHelpers.readAllIgnoreErrors(path));
}
// Return an empty database if there is no read access
return new BdfDatabase();
}
public BdfFileManager(String path) {
super(init(path));
this.path = path;
}
public void saveDatabase(String path)
{
try
{
// Get the file handler
File file = new File(path);
// Create the parent directories
file.getParentFile().mkdirs();
// Get the database file for output
FileOutputStream out = new FileOutputStream(path);
// Write the database to the file
out.write(this.serialize().getBytes());
// Close the file output stream
out.close();
}
catch(IOException e) {
return;
}
}
public void saveDatabase() {
this.saveDatabase(this.path);
}
}

View File

@ -1,78 +1,150 @@
package bdf.types;
import java.io.IOException;
import java.io.OutputStream;
import java.util.ArrayList;
import java.util.Iterator;
import bdf.data.BdfDatabase;
import bdf.util.DataHelpers;
import bdf.data.BdfStringPointer;
import bdf.data.IBdfDatabase;
import bdf.util.BdfError;
public class BdfArray implements IBdfType
public class BdfArray implements IBdfType, Iterable<BdfObject>
{
protected ArrayList<BdfObject> elements = new ArrayList<BdfObject>();
protected ArrayList<BdfObject> elements;
public BdfArray() {
BdfArray(BdfLookupTable lookupTable, BdfStringPointer ptr)
{
this.elements = new ArrayList<BdfObject>();
ptr.increment();
// [..., ...]
while(true)
{
ptr.ignoreBlanks();
if(ptr.getChar() == ']') {
ptr.increment();
return;
}
add(new BdfObject(lookupTable, ptr));
// There should be a comma after this
ptr.ignoreBlanks();
char c = ptr.getChar();
if(c == ']') {
ptr.increment();
return;
}
if(c != ',') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
ptr.increment();
ptr.ignoreBlanks();
}
}
public BdfArray(BdfDatabase data)
BdfArray(BdfLookupTable lookupTable, int size)
{
this.elements = new ArrayList<BdfObject>(size);
for(int i=0;i<size;i++) {
this.elements.add(new BdfObject(lookupTable));
}
}
BdfArray(BdfLookupTable lookupTable, IBdfDatabase data)
{
this.elements = new ArrayList<BdfObject>();
// Create an iterator value to loop over the data
int i = 0;
// Loop over the data
while(i < data.length())
while(i < data.size())
{
// Get the size of the object
int size = DataHelpers.getByteBuffer(data.getAt(i, (i+(Integer.SIZE/8)))).getInt();
int size = BdfObject.getSize(data.getPointer(i));
if(size <= 0 || i + size > data.size()) {
return;
}
// Get the object
BdfObject object = new BdfObject(data.getAt((i+(Integer.SIZE/8)), (i+(Integer.SIZE/8)+size)));
BdfObject object = new BdfObject(lookupTable, data.getPointer(i, size));
// Add the object to the elements list
elements.add(object);
// Increase the iterator by the amount of bytes
i += (Integer.SIZE/8)+size;
i += size;
}
}
@Override
public BdfDatabase serialize()
{
// Create the serialized data string
BdfDatabase serialized = new BdfDatabase();
// Loop over the elements
for(BdfObject o : elements)
{
// Convert the object to a string
BdfDatabase db = o.serialize();
// Add the serialized object to the serialized data
serialized = BdfDatabase.add(serialized, DataHelpers.serializeInt(db.length()));
serialized = BdfDatabase.add(serialized, db);
}
// Send back the serialized data
return serialized;
}
@Override
public String serializeHumanReadable()
public int serializeSeeker(int[] locations)
{
String data = "[";
int size = 0;
for(BdfObject o : elements) {
size += o.serializeSeeker(locations);
}
return size;
}
@Override
public int serialize(IBdfDatabase database, int[] locations, byte flags)
{
int pos = 0;
for(BdfObject o : elements) {
pos += o.serialize(database.getPointer(pos), locations, (byte)0);
}
return pos;
}
@Override
public void serializeHumanReadable(OutputStream stream, BdfIndent indent, int it) throws IOException
{
if(elements.size() == 0) {
stream.write("[]".getBytes());
return;
}
stream.write('[');
for(int i=0;i<elements.size();i++)
{
BdfObject o = elements.get(i);
data += o.serializeHumanReadable();
stream.write(indent.breaker.getBytes());
if(elements.size() > i+1)
{
data += ", ";
for(int n=0;n<=it;n++) {
stream.write(indent.indent.getBytes());
}
o.serializeHumanReadable(stream, indent, it + 1);
if(elements.size() > i+1) {
stream.write(", ".getBytes());
}
}
return data + "]";
stream.write(indent.breaker.getBytes());
for(int n=0;n<it;n++) {
stream.write(indent.indent.getBytes());
}
stream.write(']');
}
public BdfArray add(BdfObject o)
@ -91,9 +163,14 @@ public class BdfArray implements IBdfType
return this;
}
public BdfArray remove(int index)
{
public BdfObject remove(int index) {
BdfObject bdf = elements.get(index);
elements.remove(index);
return bdf;
}
public BdfArray remove(BdfObject bdf) {
elements.remove(bdf);
return this;
}
@ -110,4 +187,43 @@ public class BdfArray implements IBdfType
return elements.size();
}
@Override
public Iterator<BdfObject> iterator()
{
return new Iterator<BdfObject>()
{
protected int i = 0;
@Override
public boolean hasNext()
{
return elements.size() > i;
}
@Override
public BdfObject next()
{
BdfObject o = elements.get(i);
i++;
return o;
}
@Override
public void remove()
{
i-=1;
elements.remove(i);
}
};
}
@Override
public void getLocationUses(int[] locations) {
for(BdfObject element : elements) {
element.getLocationUses(locations);
}
}
}

View File

@ -0,0 +1,12 @@
package bdf.types;
public class BdfIndent
{
String indent;
String breaker;
public BdfIndent(String indent, String breaker) {
this.indent = indent;
this.breaker = breaker;
}
}

View File

@ -0,0 +1,139 @@
package bdf.types;
import java.io.OutputStream;
import java.util.ArrayList;
import bdf.data.IBdfDatabase;
import bdf.util.DataHelpers;
class BdfLookupTable implements IBdfType
{
private ArrayList<byte[]> keys;
private BdfReader reader;
BdfLookupTable(BdfReader reader)
{
this.keys = new ArrayList<byte[]>();
this.reader = reader;
}
BdfLookupTable(BdfReader reader, IBdfDatabase database)
{
this.keys = new ArrayList<byte[]>();
this.reader = reader;
for(int i=0;i<database.size();)
{
int key_size = 0xff & database.getByte(i);
i += 1;
if(i + key_size > database.size()) {
return;
}
keys.add(database.getBytes(i, key_size));
i += key_size;
}
}
int getLocation(byte[] name)
{
for(int i=0;i<keys.size();i++)
{
if(DataHelpers.bytesAreEqual(name, keys.get(i))) {
return i;
}
}
keys.add(name);
return keys.size() - 1;
}
boolean hasLocation(int location) {
return location >= 0 && location < keys.size();
}
byte[] getName(int location) {
return keys.get(location);
}
@Override
public int serialize(IBdfDatabase database, int[] locations, byte flags)
{
int upto = 0;
for(int i=0;i<locations.length;i++)
{
int loc = locations[i];
if(loc == -1) {
continue;
}
byte[] key = keys.get(i);
database.setBytes(key, upto + 1);
database.setByte(upto, (byte)key.length);
upto += key.length;
upto += 1;
}
return upto;
}
@Override
public int serializeSeeker(int[] locations)
{
int size = 0;
for(int i=0;i<locations.length;i++)
{
// Skip this key if the location is unset (the key has been culled)
int loc = locations[i];
if(loc == -1) {
continue;
}
size += keys.get(i).length;
size += 1;
}
return size;
}
public int[] serializeGetLocations()
{
int[] locations = new int[keys.size()];
int[] uses = new int[keys.size()];
int next = 0;
reader.bdf.getLocationUses(uses);
for(int i=0;i<locations.length;i++)
{
if(uses[i] > 0) {
locations[i] = next;
next += 1;
} else {
locations[i] = -1;
}
}
return locations;
}
@Override
public void serializeHumanReadable(OutputStream stream, BdfIndent indent, int it) {
}
@Override
public void getLocationUses(int[] locations) {
}
public int size() {
return keys.size();
}
}

View File

@ -1,43 +1,141 @@
package bdf.types;
import java.nio.charset.StandardCharsets;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import bdf.data.BdfDatabase;
import bdf.exception.UndefinedKeyException;
import bdf.data.BdfStringPointer;
import bdf.data.IBdfDatabase;
import bdf.util.BdfError;
import bdf.util.DataHelpers;
public class BdfNamedList implements IBdfType
{
protected class Element
{
public byte[] key;
public int key;
public BdfObject object;
}
protected ArrayList<Element> elements = new ArrayList<Element>();
protected BdfLookupTable lookupTable;
public BdfNamedList() {
BdfNamedList(BdfLookupTable lookupTable, BdfStringPointer ptr)
{
this.lookupTable = lookupTable;
ptr.increment();
// {"key": ..., "key2": ...}
while(true)
{
ptr.ignoreBlanks();
char c = ptr.getChar();
if(c == '}') {
ptr.increment();
break;
}
if(c != '"') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
String key = ptr.getQuotedString();
// There should be a colon after this
ptr.ignoreBlanks();
if(ptr.getChar() != ':') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
ptr.increment();
ptr.ignoreBlanks();
set(key, new BdfObject(lookupTable, ptr));
// There should be a comma after this
ptr.ignoreBlanks();
c = ptr.getChar();
if(c == '}') {
ptr.increment();
return;
}
if(c != ',') {
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
ptr.increment();
ptr.ignoreBlanks();
}
}
BdfNamedList(BdfLookupTable lookupTable) {
this.lookupTable = lookupTable;
}
public BdfNamedList(BdfDatabase data)
BdfNamedList(BdfLookupTable lookupTable, IBdfDatabase data)
{
this.lookupTable = lookupTable;
// Create an iterator value to loop over the data
int i = 0;
// Loop over the data
while(i < data.length())
while(i < data.size())
{
// Get the key
int key_size = DataHelpers.getByteBuffer(data.getAt(i, i+(Integer.SIZE/8))).getInt();
i += (Integer.SIZE/8);
byte[] key = data.getAt(i, i+key_size).getBytes();
// Get the object
int key_size = 0;
IBdfDatabase flag_ptr = data.getPointer(i);
int object_size = BdfObject.getSize(flag_ptr);
byte key_size_bytes = BdfObject.getParentFlags(flag_ptr);
if(object_size <= 0 || i + object_size >= data.size()) {
return;
}
BdfObject object = new BdfObject(lookupTable, data.getPointer(i, object_size));
i += object_size;
switch(key_size_bytes)
{
case 2:
key_size = 1;
break;
case 1:
key_size = 2;
break;
case 0:
key_size = 4;
break;
}
// Get the key
ByteBuffer key_buff = DataHelpers.getByteBuffer(data.getPointer(i, key_size));
int key = 0;
switch(key_size_bytes)
{
case 2:
key = 0xff & key_buff.get();
break;
case 1:
key = 0xffff & key_buff.getShort();
break;
case 0:
key = key_buff.getInt();
break;
}
if(!lookupTable.hasLocation(key)) {
return;
}
i += key_size;
int object_size = DataHelpers.getByteBuffer(data.getAt(i, i+(Integer.SIZE/8))).getInt();
i += (Integer.SIZE/8);
BdfObject object = new BdfObject(data.getAt(i, i+object_size));
// Create a new element and save some data to it
Element element = new Element();
@ -46,68 +144,117 @@ public class BdfNamedList implements IBdfType
// Add the object to the elements list
elements.add(element);
// Increase the iterator by the amount of bytes
i += object_size;
}
}
@Override
public BdfDatabase serialize()
public int serialize(IBdfDatabase database, int[] locations, byte flags)
{
// Create the serialized data string
BdfDatabase serialized = new BdfDatabase();
int pos = 0;
// Loop over the elements
for(Element o : elements)
{
// Add the serialized data to the data string
BdfDatabase data = o.object.serialize();
serialized = BdfDatabase.add(serialized, DataHelpers.serializeInt(o.key.length));
serialized = BdfDatabase.add(serialized, new BdfDatabase(o.key));
serialized = BdfDatabase.add(serialized, DataHelpers.serializeInt(data.length()));
serialized = BdfDatabase.add(serialized, data);
int location = locations[o.key];
byte size_bytes_tag;
byte size_bytes;
if(location > 65535) { // >= 2 ^ 16
size_bytes_tag = 0;
size_bytes = 4;
} else if(location > 255) { // >= 2 ^ 8
size_bytes_tag = 1;
size_bytes = 2;
} else { // < 2 ^ 8
size_bytes_tag = 2;
size_bytes = 1;
}
int size = o.object.serialize(database.getPointer(pos), locations, size_bytes_tag);
int offset = pos + size;
byte[] bytes = DataHelpers.serializeInt(location);
for(int i=0;i<size_bytes;i++) {
database.setByte(i + offset, bytes[i - size_bytes + 4]);
}
pos += size + size_bytes;
}
// Send back the serialized data
return serialized;
return pos;
}
@Override
public String serializeHumanReadable()
public int serializeSeeker(int[] locations)
{
String data = "{";
int size = 0;
for(Element o : elements)
{
int location = locations[o.key];
if(location > 65535) { // >= 2 ^ 16
size += 4;
} else if(location > 255) { // >= 2 ^ 8
size += 2;
} else { // < 2 ^ 8
size += 1;
}
size += o.object.serializeSeeker(locations);
}
return size;
}
@Override
public void serializeHumanReadable(OutputStream stream, BdfIndent indent, int it) throws IOException
{
if(elements.size() == 0) {
stream.write("{}".getBytes());
return;
}
stream.write('{');
for(int i=0;i<elements.size();i++)
{
Element e = elements.get(i);
data += DataHelpers.serializeString(new String(e.key, StandardCharsets.UTF_8));
data += ": ";
data += e.object.serializeHumanReadable();
stream.write(indent.breaker.getBytes());
if(elements.size() > i+1)
{
data += ", ";
for(int n=0;n<=it;n++) {
stream.write(indent.indent.getBytes());
}
stream.write((DataHelpers.serializeString(new String(lookupTable.getName(e.key))) + ": ").getBytes());
e.object.serializeHumanReadable(stream, indent, it + 1);
if(elements.size() > i+1) {
stream.write(", ".getBytes());
}
}
return data + "}";
stream.write(indent.breaker.getBytes());
for(int n=0;n<it;n++) {
stream.write(indent.indent.getBytes());
}
stream.write('}');
}
public BdfObject get(String key)
public BdfObject get(int key)
{
// Get the object to send back
BdfObject object = null;
// Convert the key to bytes
byte[] key_bytes = key.getBytes();
// Loop over the elements
for(Element e : elements)
{
// Is this the element key
if(DataHelpers.bytesAreEqual(e.key, key_bytes))
if(e.key == key)
{
// Set the object
object = e.object;
@ -117,15 +264,18 @@ public class BdfNamedList implements IBdfType
}
}
// Raise an error
throw new UndefinedKeyException(key);
// Get a bdf object
BdfObject o = new BdfObject(lookupTable);
// Set the bdf object
this.set(key, o);
// Send back the object
return o;
}
public void remove(String key)
public BdfObject remove(int key)
{
// Convert the key to bytes
byte[] key_bytes = key.getBytes();
// Loop over the elements
for(int i=0;i<elements.size();i++)
{
@ -133,58 +283,50 @@ public class BdfNamedList implements IBdfType
Element e = elements.get(i);
// Is the specified key the same as the elements key
if(DataHelpers.bytesAreEqual(e.key, key_bytes))
{
// Delete this element
elements.remove(i);
// Exit out of the function, prevent NullPointException
return;
if(e.key == key) {
return elements.remove(i).object;
}
}
// Raise an error
throw new UndefinedKeyException(key);
// Send back nothing
return null;
}
public void set(String key, BdfObject object)
public BdfNamedList set(int key, BdfObject object)
{
// Convert the key to bytes
byte[] key_bytes = key.getBytes();
// Loop over the elements, does it already exist
for(Element e : elements)
{
// Is the key here the same as the specified key
if(DataHelpers.bytesAreEqual(e.key, key_bytes))
if(e.key == key)
{
// Set the new object
e.object = object;
// Exit out of the function, don't add another object
return;
return this;
}
}
// Create a new element object
Element e = new Element();
e.key = key_bytes;
e.object = object;
e.key = key;
// Add the new element object to the elements list
elements.add(e);
// Send this class back
return this;
}
public boolean contains(String key)
public boolean contains(int key)
{
// Convert the key to bytes
byte[] key_bytes = key.getBytes();
// Loop over the elements
for(Element e : elements)
{
// Is the elements key the same as the specified key
if(DataHelpers.bytesAreEqual(e.key, key_bytes))
if(e.key == key)
{
// Send back true to say the element was found
return true;
@ -195,25 +337,49 @@ public class BdfNamedList implements IBdfType
return false;
}
public String[] getKeys()
public int[] getKeys()
{
// Get the keys to send back
String[] keys = new String[elements.size()];
int[] keys = new int[elements.size()];
// Loop over the elements
for(int i=0;i<elements.size();i++)
{
// Get the element
Element e = elements.get(i);
keys[i] = new String(e.key, StandardCharsets.UTF_8);
keys[i] = e.key;
}
// Return the list of keys as strings
return keys;
}
public boolean contains(String key) {
return contains(lookupTable.getLocation(key.getBytes()));
}
public BdfNamedList set(String key, BdfObject object) {
return set(lookupTable.getLocation(key.getBytes()), object);
}
public BdfObject remove(String key) {
return remove(lookupTable.getLocation(key.getBytes()));
}
public BdfObject get(String key) {
return get(lookupTable.getLocation(key.getBytes()));
}
public int size() {
return elements.size();
}
@Override
public void getLocationUses(int[] locations)
{
for(Element e : elements) {
locations[e.key] += 1;
e.object.getLocationUses(locations);
}
}
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,182 @@
package bdf.types;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.ByteBuffer;
import bdf.data.BdfDatabase;
import bdf.data.IBdfDatabase;
import bdf.util.DataHelpers;
public class BdfReader
{
protected BdfLookupTable lookupTable;
protected BdfObject bdf;
private void initNew() {
lookupTable = new BdfLookupTable(this);
bdf = new BdfObject(lookupTable);
}
public BdfReader() {
initNew();
}
public BdfReader(byte[] database) {
this(new BdfDatabase(database));
}
public BdfReader(IBdfDatabase database)
{
try
{
if(database.size() == 0) {
initNew();
return;
}
int upto = 0;
IBdfDatabase flag_ptr = database.getPointer(upto);
byte lookupTable_size_tag = BdfObject.getParentFlags(flag_ptr);
byte lookupTable_size_bytes = 0;
switch(lookupTable_size_tag)
{
case 0:
lookupTable_size_bytes = 4;
break;
case 1:
lookupTable_size_bytes = 2;
break;
case 2:
lookupTable_size_bytes = 1;
break;
}
if(lookupTable_size_bytes > database.size()) {
initNew();
return;
}
// Get the rest of the data
int bdf_size = BdfObject.getSize(flag_ptr);
if(bdf_size + lookupTable_size_bytes > database.size()) {
initNew();
return;
}
IBdfDatabase database_bdf = database.getPointer(upto, bdf_size);
upto += bdf_size;
// Get the lookup table
ByteBuffer lookupTable_size_buff = DataHelpers.getByteBuffer(database.getPointer(upto, lookupTable_size_bytes));
int lookupTable_size = 0;
switch(lookupTable_size_tag)
{
case 0:
lookupTable_size = lookupTable_size_buff.getInt();
break;
case 1:
lookupTable_size = 0xffff & lookupTable_size_buff.getShort();
break;
case 2:
lookupTable_size = 0xff & lookupTable_size_buff.get();
break;
}
if(lookupTable_size + lookupTable_size_bytes + bdf_size > database.size()) {
initNew();
return;
}
lookupTable = new BdfLookupTable(this, database.getPointer(lookupTable_size_bytes + upto, lookupTable_size));
bdf = new BdfObject(lookupTable, database_bdf);
}
catch(IndexOutOfBoundsException e) {
initNew();
}
}
public BdfDatabase serialize()
{
int[] locations = lookupTable.serializeGetLocations();
int bdf_size = bdf.serializeSeeker(locations);
int lookupTable_size = lookupTable.serializeSeeker(locations);
int lookupTable_size_bytes = 0;
byte lookupTable_size_tag = 0;
if(lookupTable_size > 65535) { // >= 2 ^ 16
lookupTable_size_tag = 0;
lookupTable_size_bytes = 4;
} else if(lookupTable_size > 255) { // >= 2 ^ 8
lookupTable_size_tag = 1;
lookupTable_size_bytes = 2;
} else { // < 2 ^ 8
lookupTable_size_tag = 2;
lookupTable_size_bytes = 1;
}
int upto = 0;
int database_size = bdf_size + lookupTable_size + lookupTable_size_bytes;
BdfDatabase database = new BdfDatabase(database_size);
bdf.serialize(database.getPointer(upto, bdf_size), locations, lookupTable_size_tag);
upto += bdf_size;
byte[] bytes = DataHelpers.serializeInt(lookupTable_size);
for(int i=0;i<lookupTable_size_bytes;i++) {
database.setByte(i + upto, bytes[i - lookupTable_size_bytes + 4]);
}
lookupTable.serialize(database.getPointer(upto + lookupTable_size_bytes, lookupTable_size), locations, (byte)0);
return database;
}
public BdfObject getObject() {
return bdf;
}
public BdfObject resetObject() {
bdf = new BdfObject(lookupTable);
return bdf;
}
public String serializeHumanReadable(BdfIndent indent)
{
ByteArrayOutputStream stream = new ByteArrayOutputStream();
try {
bdf.serializeHumanReadable(stream, indent, 0);
return stream.toString();
}
catch(IOException e) {
return "undefined";
}
}
public String serializeHumanReadable() {
return serializeHumanReadable(new BdfIndent("", ""));
}
public void serializeHumanReadable(OutputStream stream, BdfIndent indent) throws IOException
{
bdf.serializeHumanReadable(stream, indent, 0);
stream.write('\n');
stream.flush();
}
public void serializeHumanReadable(OutputStream stream) throws IOException {
serializeHumanReadable(stream, new BdfIndent("", ""));
}
}

View File

@ -0,0 +1,29 @@
package bdf.types;
import bdf.data.BdfStringPointer;
import bdf.util.BdfError;
public class BdfReaderHuman extends BdfReader
{
public BdfReaderHuman(String data)
{
BdfStringPointer ptr = new BdfStringPointer(data.toCharArray(), 0);
ptr.ignoreBlanks();
bdf = new BdfObject(lookupTable, ptr);
try {
ptr.ignoreBlanks();
}
catch(BdfError e) {
if(e.getType() == BdfError.ERROR_END_OF_FILE) {
return;
} else {
throw e;
}
}
throw BdfError.createError(BdfError.ERROR_SYNTAX, ptr);
}
}

View File

@ -2,15 +2,25 @@ package bdf.types;
public class BdfTypes
{
public static final byte BOOLEAN = 0;
public static final byte INTEGER = 1;
public static final byte LONG = 2;
public static final byte SHORT = 3;
public static final byte BYTE = 4;
public static final byte DOUBLE = 5;
public static final byte FLOAT = 6;
public static final byte STRING = 7;
public static final byte ARRAY = 8;
public static final byte NAMED_LIST = 9;
public static final byte EMPTY = 10;
public static final byte UNDEFINED = 0;
public static final byte BOOLEAN = 1;
public static final byte INTEGER = 2;
public static final byte LONG = 3;
public static final byte SHORT = 4;
public static final byte BYTE = 5;
public static final byte DOUBLE = 6;
public static final byte FLOAT = 7;
public static final byte STRING = 8;
public static final byte ARRAY = 9;
public static final byte NAMED_LIST = 10;
public static final byte ARRAY_BOOLEAN = 11;
public static final byte ARRAY_INTEGER = 12;
public static final byte ARRAY_LONG = 13;
public static final byte ARRAY_SHORT = 14;
public static final byte ARRAY_BYTE = 15;
public static final byte ARRAY_DOUBLE = 16;
public static final byte ARRAY_FLOAT = 17;
}

View File

@ -1,9 +1,15 @@
package bdf.types;
import bdf.data.BdfDatabase;
import java.io.IOException;
import java.io.OutputStream;
public interface IBdfType
import bdf.data.IBdfDatabase;
interface IBdfType
{
public BdfDatabase serialize();
public String serializeHumanReadable();
void getLocationUses(int[] locations);
int serialize(IBdfDatabase database, int[] locations, byte flags);
int serializeSeeker(int[] locations);
void serializeHumanReadable(OutputStream stream, BdfIndent indent, int it) throws IOException;
}

108
src/bdf/util/BdfError.java Normal file
View File

@ -0,0 +1,108 @@
package bdf.util;
import bdf.data.BdfStringPointer;
public class BdfError extends RuntimeException
{
private static final long serialVersionUID = -8731016151496842808L;
private static final String[] ERRORS = {
"Syntax error",
"End of file",
"Unescaped comment",
"Unescaped string",
"Number format error",
};
public static final int ERROR_SYNTAX = 0;
public static final int ERROR_END_OF_FILE = 1;
public static final int ERROR_UNESCAPED_COMMENT = 2;
public static final int ERROR_UNESCAPED_STRING = 3;
public static final int ERROR_NUMBER = 4;
public static BdfError createError(int errorID, BdfStringPointer ptr)
{
String error = ERRORS[errorID];
char[] array = ptr.getDataCharArray();
int location = ptr.getDataLocation();
int start_of_line = 0;
int line = 0;
int at = 0;
for(int i=0;i<location;i++)
{
if(array[i] == '\n') {
start_of_line = i + 1;
line += 1;
at = 0;
continue;
}
at += 1;
}
int line_size = 0;
String spacer = "";
for(int i=start_of_line;i<array.length;i++)
{
if(array[i] == '\n') {
break;
}
line_size += 1;
if(i == array.length - 1) {
break;
}
if(i < location)
{
if(array[i] == '\t') {
spacer += "\t";
continue;
}
spacer += " ";
}
}
char[] line_chars = new char[line_size];
for(int i=0;i<line_size;i++) {
line_chars[i] = array[start_of_line + i];
}
String message = "";
String error_short = error + " " + (line + 1) + ":" + (at + 1);
message += error_short + "\n";
message += new String(line_chars) + "\n";
message += spacer;
message += "^";
BdfError bdf_error = new BdfError(message);
bdf_error.error_short = error_short;
bdf_error.type = errorID;
return bdf_error;
}
private String error_short;
private int type;
public int getType() {
return type;
}
public String getErrorShort() {
return error_short;
}
private BdfError(String message) {
super(message);
}
}

View File

@ -3,10 +3,15 @@ package bdf.util;
import java.nio.ByteBuffer;
import bdf.data.BdfDatabase;
import bdf.data.IBdfDatabase;
public class DataHelpers
{
public static ByteBuffer getByteBuffer(BdfDatabase db) {
private static final char[] HEX = {
'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'
};
public static ByteBuffer getByteBuffer(IBdfDatabase db) {
return ByteBuffer.wrap(db.getBytes());
}
@ -14,11 +19,11 @@ public class DataHelpers
return new BdfDatabase(buffer.array());
}
public static BdfDatabase serializeInt(int value)
public static byte[] serializeInt(int value)
{
ByteBuffer buffer = ByteBuffer.allocate(Integer.SIZE/8);
ByteBuffer buffer = ByteBuffer.allocate(4);
buffer.putInt(value);
return getDatabase(buffer);
return buffer.array();
}
public static boolean bytesAreEqual(byte[] b1, byte[] b2)
@ -37,17 +42,16 @@ public class DataHelpers
return true;
}
public static String replaceInString(String string, byte find, String replace)
public static String replaceInString(String string, char find, String replace)
{
// Convert the string to bytes
byte[] string_b = string.getBytes();
String string_modified = new String();
// Loop over the string
for(int i=0;i<string_b.length;i++)
for(int i=0;i<string.length();i++)
{
// Is the byte to find the byte at this part of the string
if(find == string_b[i])
if(find == string.charAt(i))
{
// Add the data to replace to the string
string_modified += replace;
@ -64,22 +68,110 @@ public class DataHelpers
return string_modified;
}
public static String replaceInString(String string, char find, String replace) {
return replaceInString(string, (byte)find, replace);
public static char[] replaceInCharArray(char[] chars, char find, String replace)
{
char[] replace_chars = replace.toCharArray();
int replace_size = replace.length();
int size = 0;
// Get the new size of the char array
for(int i=0;i<chars.length;i++) {
if(chars[i] == find) {
size += replace_size;
} else {
size += 1;
}
}
char[] chars_modified = new char[size];
int upto = 0;
// Replace the contents of the char array
for(int i=0;i<chars.length;i++)
{
if(chars[i] == find)
{
for(int j=0;j<replace_size;j++) {
chars_modified[upto+j] = replace_chars[j];
}
upto += replace_size;
}
else
{
chars_modified[upto] = chars[i];
upto += 1;
}
}
return chars_modified;
}
public static String serializeString(String string)
{
// Serialize the string
String serialized = string;
char[] string_chars = string.toCharArray();
// Replace some parts of the string
serialized = replaceInString(serialized, '"', "\\\"");
string_chars = replaceInCharArray(string_chars, '\\', "\\\\");
string_chars = replaceInCharArray(string_chars, '"', "\\\"");
string_chars = replaceInCharArray(string_chars, '\n', "\\n");
string_chars = replaceInCharArray(string_chars, '\t', "\\t");
// Replace all the unreadable parts of the string
{
int size = 0;
for(int i=0;i<string_chars.length;i++)
{
char c = string_chars[i];
if(c < 0x20 || (c > 0x7e && c < 0xa1) || c == 0xad)
{
// Will be in the format \u0000
size += 6;
}
else
{
size += 1;
}
}
char[] chars = new char[size];
int upto = 0;
for(int i=0;i<string_chars.length;i++)
{
char c = string_chars[i];
if(c < 0x20 || (c > 0x7e && c < 0xa1) || c == 0xad)
{
// Will be in the format \u0000
chars[upto] = '\\';
chars[upto+1] = 'u';
chars[upto+2] = HEX[(c & 0xf000) >> 12];
chars[upto+3] = HEX[(c & 0x0f00) >> 8];
chars[upto+4] = HEX[(c & 0x00f0) >> 4];
chars[upto+5] = HEX[(c & 0x000f)];
upto += 6;
}
else
{
chars[upto] = string_chars[i];
upto += 1;
}
}
string_chars = chars;
}
// Add quotes to the string
serialized = "\"" + serialized + "\"";
string = "\"" + new String(string_chars) + "\"";
// Return the serialized string
return serialized;
return string;
}
}

View File

@ -1,59 +0,0 @@
package bdf.util;
import java.io.FileInputStream;
import java.io.IOException;
public class FileHelpers
{
public static byte[] readAll(FileInputStream in)
{
try
{
// Get bytes to return
int available = in.available();
byte[] bytes = new byte[available];
// Loop over the available bytes
for(int i=0;i<available;i++)
{
// Add the next byte from the stream to the bytes
bytes[i] = (byte) in.read();
}
// Send the bytes collected from the file stream back
return bytes;
}
catch (IOException e)
{
// Throw the IOException as a runtime exception
throw new RuntimeException(e);
}
}
public static byte[] readAll(String path) throws IOException
{
// Create the file input stream
FileInputStream in = new FileInputStream(path);
// Load all of its data
byte[] data = readAll(in);
// Close the file input stream
in.close();
// Send back the data
return data;
}
public static byte[] readAllIgnoreErrors(String path)
{
try {
return readAll(path);
}
catch(IOException e) {
return new byte[0];
}
}
}

36
src/tests/Tests.java Executable file
View File

@ -0,0 +1,36 @@
package tests;
import java.io.FileInputStream;
import java.io.IOException;
import bdf.data.IBdfDatabase;
import bdf.types.BdfReader;
import bdf.types.BdfReaderHuman;
public class Tests
{
public static void displayHex(IBdfDatabase db)
{
char[] hex_chars = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'};
System.out.print("Size: " + db.size() + ", Hex: ");
for(int i=0;i<db.size();i++)
{
int b = db.getByte(i);
if(b < 0) b += 128;
System.out.print(hex_chars[b / 16]);
System.out.print(hex_chars[b % 16]);
System.out.print(' ');
}
System.out.println();
}
public static void main(String[] args) throws IOException
{
new BdfReaderHuman("int(NaND)");
}
}