[vxl2txt, txt2vxl] File format converters (new: c++ version)

External additional handy features.
11 posts Page 1 of 1
Stiivais
Deuce
Posts: 16
Joined: Sat Oct 25, 2014 8:49 am


VXL2TXT (v2, stable)
Here's a little command-line tool i made in pascal, that takes a .vxl file and converts its binary byte values into a text file with a customizable delimiter. Pretty fast, converts your average map in less than a second. Currently doesn't do anything if the process fails. Probably could be used as a part of a toolchain to interpret map data for whatever reasons... another custom server implementation, anyone?

Command line options (each next option requires the previous to be present, that's what the 0 is for):
1. input - none, 0 or string - the input file, filename+extension, can be absolute path, if none/0 then 'input.vxl'
2. output - none, 0 or string - the output file, filename+extension, can be absolute path, if none/0 then 'output.txt'
3. delimiter - none, 0 or char (string works, not recommended) - the value delimiter, if none/0 then ',' (comma, used in .csv)
4. wait - none, 'wait' (no quotes) - if present, makes program wait for user to press any key to quit

Example usage:
  • vxl2txt.exe mycoolmap.vxl nowistext.txt . wait
    this will take the file mycoolmap.vxl and process it into nowistext.txt with delimiter '.' and will wait for user input before closing
  • vxl2txt.exe c:\aos\maps\mapfile.vxl 0 -
    this will take the file at c:\... and process it into output.txt (in the same directory as .exe) with delimiter '-', and will close after done
  • vxl2txt.exe 0 0 0 wait
    this will use the default parameters (input.vlx, output.txt, comma delimited), but will wait for user input before closing
Download (v2) (182KB): https://www.dropbox.com/s/7d1oxmhemmn1l ... t.exe?dl=0

Source code (v2) (Pascal, commented, 71 line): http://pastebin.com/vw8HxiXE


TXT2VXL CPP (v1, stable)
After being unhappy with the speed TXT2VXL ran at, i decided to rewrite it in C. That plan failed miserably because i am incapable of wrapping my head around memory management in that language (even with the help of LeCom), so i switched over to C++. That went a bit better, so i bring you TXT2VXL CPP. I still found that programming in Pascal is easier for me, and the algorithm i used this time might not be the most efficient, but the results speak for themselves - TXT2VXL processed my map in around 8 seconds, whereas TXT2VXL CPP needed only 2, the C++ version being 4 times faster. Also, the command line parameters are implemented in the exact same way as for the tool below, although the way results are displayed is slightly different. The files generated are exactly the same, as checked with a Notepad++ plugin.

Download (v1) (500 KB): https://www.dropbox.com/s/ik5x4yaw4jk4z ... p.exe?dl=0

Source code (v1) (C++, commented, 103 lines): http://pastebin.com/9Ab48gt2


TXT2VXL (v5.1, stable)
I figured that the previous tool would be pretty much useless without a way to get back to .vxl file, so after some initial difficulties, i made one. This does the exact opposite of the tool above - it takes a .txt file with byte values in ascii format, separated by a delimiter, and converts them into bytes stored in binary. The good part is that the delimiter is detected automatically, although only a single character is supported because of this. It is certainly a lot slower than the first tool, but still converts a map in less than 10 seconds.

Command line options (each next option requires the previous to be present, that's what the 0 is for):
1. input - none, 0 or string - the input file, filename+extension, can be absolute path, if none/0 then 'output.txt'
2. output - none, 0 or string - the output file, filename+extension, can be absolute path, if none/0 then 'output.vxl'
3. wait - none, 'wait' (no quotes) - if present, makes program wait for user to press any key to quit

Example usage:
  • same as above, just without the delimiter parameter
Download (v5.1) (185KB): https://www.dropbox.com/s/m0qxyn0e5ameg ... l.exe?dl=0

Source code (v5.1) (Pascal, commented, 108 lines): http://pastebin.com/nZpMypr5


Test if it works.
If you aren't very confident of my programming skills (i sure as hell am not), then feel free to grab a map file and put it in the directory of the executable files, then convert it back and forth. I've even made this easy for you - just rename your map file "input.vxl", then run vxl2txt and once it's done, txt2vxl. You will get 2 files - output.txt, which is the delimited text format, and output.vxl, which should be perfectly identical to your .vxl map file.

Here's a screenshot (click for full size):
Image


Bugs.
  • The file size readings given out by the application seem a bit off.
Todo.
  • Make something to convert .vxl into an array of voxels.
  • Convert these utilities to .dll format.
Last edited by Stiivais on Thu Oct 30, 2014 4:17 pm, edited 15 times in total.
Lincent
Veterans
Veterans
Posts: 693
Joined: Wed Mar 27, 2013 9:47 pm


Did you test a map?
What was the output?
Stiivais
Deuce
Posts: 16
Joined: Sat Oct 25, 2014 8:49 am


Lincent wrote:
Did you test a map?
What was the output?
Yes, i did, it was as expected - a list of byte values in ascii separated by the intended symbol. As for the correctness of the format, i doubt that anything could have gone wrong, since pretty much no operations apart from binary conversion are done.
LeCom
Post Demon
Post Demon
Posts: 866
Joined: Sat May 24, 2014 8:07 am


We live in the 21st century. Nobody is using pascal anymore. It's a good idea to use C: it's pretty similar (kind of evolved from Pascal as I heard), still very popular, and it would help you a lot with your problem because C is exactly where you don't have such problems, or can track the problems down easily. If you want anybody to contribute to your project or help you with problems, you shouldn't use a rare language.
Stiivais
Deuce
Posts: 16
Joined: Sat Oct 25, 2014 8:49 am


LeCom wrote:
We live in the 21st century. Nobody is using pascal anymore. It's a good idea to use C: it's pretty similar (kind of evolved from Pascal as I heard), still very popular, and it would help you a lot with your problem because C is exactly where you don't have such problems, or can track the problems down easily. If you want anybody to contribute to your project or help you with problems, you shouldn't use a rare language.
Point taken, but i solved the problem. About to hunt down 1 more bug and then release a functional version. Pascal is the first language i started learning, so most of my knowledge is about it. Of course, i also started learning C++ (C isn't really object oriented, C# seems to specific, leaving me with that rather difficult language) and have a bunch of other IDEs and books on my HDD, but i don't want to spread out too fast and actually learn things properly before moving onwards (in case of pascal it is reading 3 books i downloaded that detail various programming concepts).

To be honest, as long as i produce .exe or .dll files, the language i use doesn't matter that much, unless you care about performance - then again, you guys have a server written in python, an interpreted language, that is painfully slow compared to Pascal and all derivatives of C.
longbyte1
Deuced Up
Posts: 336
Joined: Sun Jul 21, 2013 7:27 pm


LeCom wrote:
We live in the 21st century. Nobody is using pascal anymore. It's a good idea to use C: it's pretty similar (kind of evolved from Pascal as I heard), still very popular, and it would help you a lot with your problem because C is exactly where you don't have such problems, or can track the problems down easily. If you want anybody to contribute to your project or help you with problems, you shouldn't use a rare language.
Pascal is great for small little programs, it wouldn't take much effort to convert it to Python.
LeCom
Post Demon
Post Demon
Posts: 866
Joined: Sat May 24, 2014 8:07 am


Stiivais wrote:
LeCom wrote:
We live in the 21st century. Nobody is using pascal anymore. It's a good idea to use C: it's pretty similar (kind of evolved from Pascal as I heard), still very popular, and it would help you a lot with your problem because C is exactly where you don't have such problems, or can track the problems down easily. If you want anybody to contribute to your project or help you with problems, you shouldn't use a rare language.
Point taken, but i solved the problem. About to hunt down 1 more bug and then release a functional version. Pascal is the first language i started learning, so most of my knowledge is about it. Of course, i also started learning C++ (C isn't really object oriented, C# seems to specific, leaving me with that rather difficult language) and have a bunch of other IDEs and books on my HDD, but i don't want to spread out too fast and actually learn things properly before moving onwards (in case of pascal it is reading 3 books i downloaded that detail various programming concepts).

To be honest, as long as i produce .exe or .dll files, the language i use doesn't matter that much, unless you care about performance - then again, you guys have a server written in python, an interpreted language, that is painfully slow compared to Pascal and all derivatives of C.
C isn't worse than C++, just because it's not object-oriented. Concerning PySnip, it's server software that requires a minimum of calculations, and it contains much C code.
C is low-level, which makes it good for optimizing things and you have full control over each byte of RAM and almost each CPU operation, that's why I recommended it for implementing a .vxl reader.
If you ignore C, and want to use Pascal, go on. I'm interested into how easy it is to implement something like this in Pascal.
Stiivais
Deuce
Posts: 16
Joined: Sat Oct 25, 2014 8:49 am


LeCom wrote:
...
I didn't say that it's worse, just that i tought that i should start learning programming with a language that is object oriented, or has such capabilities - from what little i've read, it appears that C is more procedural in nature.
However, after trying to achieve threading in Pascal, i think i'll just switch over to C - Pascal isn't very intuitive once things get more complicated, and as you say, it's not very popular/useful, past learning basic programming concepts. But once i post crappy, badly thought out code that makes you cringe, please keep in mind that this is still very much a learning experience for me (feel free to correct me and scold bad practices, though)...

Also, does the python server use .DLLs written in C or something? I don't really understand how else python could use C code otherwise, would you be kind enough to explain?

Lastly, do you think that "C Primer Plus" is a good book to learn the language?

Oh, and reading these words after mucking around in Pascal for a while (probably due to my own incompetence, or perhaps the lack of readable documentation and a serious lack of books on the internet) is quite relieving:
ImageImage

Edit: C doesn't have a string data type. What the actual f**k?
LeCom
Post Demon
Post Demon
Posts: 866
Joined: Sat May 24, 2014 8:07 am


Stiivais wrote:
Also, does the python server use .DLLs written in C or something? I don't really understand how else python could use C code otherwise, would you be kind enough to explain?
PySnip uses Cython, which allows you to call pieces of C code from Python. On Windows, the C code is saved together with the python code in the .exe file. Don't know how it is on *nix, but it should be similar.
But Python isn't even that slow. You just can't do insanely heavy calculations with the speed of C.
Stiivais wrote:
Lastly, do you think that "C Primer Plus" is a good book to learn the language?
I don't know any programming books at all. I think the best way is to learn it on yourself - find some simple C code, try to understand what it does and google stuff you don't know or can't understand. But if you are a real noob, a book may help you.
Stiivais wrote:
Oh, and reading these words after mucking around in Pascal for a while (probably due to my own incompetence, or perhaps the lack of readable documentation and a serious lack of books on the internet) is quite relieving:
ImageImage
That text is pretty true. Tho' you skipped the most important part that explains the dangers of C. Since it doesn't have actual error handling, C programs crash on any occasion. GDB can catch these crashes and show you the line of code where it crashed, but sometimes its output is fucked up (memory corruption) or that line is clean and the error is somewhere else (especially when you have float calculations, because the x86 FPU likes to return infinite values or NaN's when you try to do something invalid).
So C is best for simple and short code where you need to have full control over everything, what is your case. If it gets complex, you should probably try something higher-level (Pascal if you want. Tho' I recommend Python).
Stiivais wrote:
Edit: C doesn't have a string data type. What the actual f**k?
Actually C does have a string data type. It's just technically an array of characters instead of an object as you have them usually.
Stiivais
Deuce
Posts: 16
Joined: Sat Oct 25, 2014 8:49 am


LeCom wrote:
Stiivais wrote:
...
PySnip uses Cython, which allows you to call pieces of C code from Python. On Windows, the C code is saved together with the python code in the .exe file. Don't know how it is on *nix, but it should be similar.
But Python isn't even that slow. You just can't do insanely heavy calculations with the speed of C.
Cython actually sounds pretty interesting... But from the couple of benchmarks i've seen, it seems that pure Python is unsuited for anything dealing with large numbers, which is exactly what i'm trying to do.
LeCom wrote:
Stiivais wrote:
...
I don't know any programming books at all. I think the best way is to learn it on yourself - find some simple C code, try to understand what it does and google stuff you don't know or can't understand. But if you are a real noob, a book may help you.
That's an interesting approach, but wouldn't that make you miss how certain commands work internally? Sure, the first chapters of the book i'm reading are a bit deja-vu when compared to a Pascal book, but there certainly are important things that might be hard to notice otherwise.
LeCom wrote:
Stiivais wrote:
...
That text is pretty true. Tho' you skipped the most important part that explains the dangers of C. Since it doesn't have actual error handling, C programs crash on any occasion. GDB can catch these crashes and show you the line of code where it crashed, but sometimes its output is fucked up (memory corruption) or that line is clean and the error is somewhere else (especially when you have float calculations, because the x86 FPU likes to return infinite values or NaN's when you try to do something invalid).
So C is best for simple and short code where you need to have full control over everything, what is your case. If it gets complex, you should probably try something higher-level (Pascal if you want. Tho' I recommend Python).
Will definitely keep this in mind, although i kinda don't feel all that comfortable with using python because of the language version issues (2 or 3), the fact that it's interpreted or just that i somehow mess up setting up anything that's not PortablePython (pre-installed libraries like py2exe and a couple of IDEs)...
LeCom wrote:
Stiivais wrote:
...
Actually C does have a string data type. It's just technically an array of characters instead of an object as you have them usually.
Time to learn memory management so i can allocate or free memory every time i need to change what's written in a string. Or perhaps just have a fixed size array and have a character that's considered the end of the string. Or, you know, use a library, because there's probably one for this.

Either way, your help is really appreciated, it's nice to have someone who can recommend things to you, give their opinions and/or a helping hand. Green_Happy1
LeCom
Post Demon
Post Demon
Posts: 866
Joined: Sat May 24, 2014 8:07 am


Stiivais wrote:
LeCom wrote:
Stiivais wrote:
...
I don't know any programming books at all. I think the best way is to learn it on yourself - find some simple C code, try to understand what it does and google stuff you don't know or can't understand. But if you are a real noob, a book may help you.
That's an interesting approach, but wouldn't that make you miss how certain commands work internally? Sure, the first chapters of the book i'm reading are a bit deja-vu when compared to a Pascal book, but there certainly are important things that might be hard to notice otherwise.
Just look up a list of keywords, operators, a description of the whole syntax and the stdlib (http://www.cplusplus.com/reference/clibrary/)
Stiivais wrote:
LeCom wrote:
Stiivais wrote:
...
That text is pretty true. Tho' you skipped the most important part that explains the dangers of C. Since it doesn't have actual error handling, C programs crash on any occasion. GDB can catch these crashes and show you the line of code where it crashed, but sometimes its output is fucked up (memory corruption) or that line is clean and the error is somewhere else (especially when you have float calculations, because the x86 FPU likes to return infinite values or NaN's when you try to do something invalid).
So C is best for simple and short code where you need to have full control over everything, what is your case. If it gets complex, you should probably try something higher-level (Pascal if you want. Tho' I recommend Python).
Will definitely keep this in mind, although i kinda don't feel all that comfortable with using python because of the language version issues (2 or 3), the fact that it's interpreted or just that i somehow mess up setting up anything that's not PortablePython (pre-installed libraries like py2exe and a couple of IDEs)...
You aren't comfortable with Python being interpreted? You didn't see its advantages - you can do really interesting stuff, like accessing variables by name, making a running program interpret commands from your keyboard or a file, importing .py files in realtime, etc.
If you have problems with setting up libraries, just try to use what the default library offers you, and only add an extension if there's no alternative.
Stiivais wrote:
LeCom wrote:
Actually C does have a string data type. It's just technically an array of characters instead of an object as you have them usually.
Time to learn memory management so i can allocate or free memory every time i need to change what's written in a string. Or perhaps just have a fixed size array and have a character that's considered the end of the string. Or, you know, use a library, because there's probably one for this.
C coders often just allocate a fixed size array ('\0' is always the end of a C string btw), and cut strings that are copied in there.
11 posts Page 1 of 1
Return to “Add-ons & Tools”

Who is online

Users browsing this forum: No registered users and 1 guest