Evaluator rewrite, diffing view and more

Documentation

A new documentation for ImHex (mainly the pattern language right now) is in the works and can be found here: https://imhex.werwolv.net/docs. It is a lot more detailed than the one found on the Wiki here and contains information about all the new features as well. Check it out!

Changelog

Additions

• Added support for loading multiple files/data sources simultaneously
• Added Binary Diffing view
• Added Quit ImHex option to File menu
• Added Follow system theme option
• Added Chinese (Simplified) Localization. Many thanks to @xtexChooser

Pattern Language

• Rewrote entire evaluation engine
  • Made Evaluator a lot simpler and faster
  • Function parameters now require a type definition
  • Strings may now be passed to functions using the str type
  • Added check to prevent huge arrays from being generated (can be overridden with a pragma)
  • Added cast expression
  • Added std::format
    • It and std::print now use libfmt / C++'s std::format syntax
  • Allow passing variables of custom types to functions
  • Added multi-variable declaration for functions
  • Many more under-the-hood changess
• [[format]] attribute to override the way values are displayed
• Added Unit tests
  • This will hopefully help with things breaking in the pattern language randomly
• Unnecessary semicolons are now ignored
• Added std::mem::read_string built-in function

Improvements

• **On Windows, resources are now placed in the AppData folder in addition to next to the executable``
• Fixed format-security warnings. Thanks to @russkel
• Moved all Hex Editor settings to the settings menu
• Restructured the settings menu into multiple tabs
• Fixed footer items jumping around a lot when updating
• Removed View suffix from all View names in the Views menu. Thanks to @xtexChooser
• Fixed pattern language evaluator "lagging" behind when Automatic evaluation is active

Bug fixes

• Fixed settings not being initialized correctly sometimes and causing crashes
  • No more crashes when settings have been set to invalid values
• Fixed recent files not updating properly
• Fixed Resize File option crashing
• Fixed crash when creating arrays with negative size.
• Fixed parent keyword not working correctly
• Fixed constants store page not getting cleared correctly on reload
• Fixed feedback button URL
• Fixed padding not working correctly
• Fixed enums not counting properly
• Fixed bitfields not respecting endian setting
• Fixed language selection combo box crashing in some circumstances
• Fixed settings menu closing when changing language
• Fixed Regex replacer tool
• Fixed TTY console crashing if no COM ports are available. Thanks to @xtexChooser

If you like my work, please consider supporting me on GitHub Sponsors, Patreon or PayPal. Thanks a lot!

Pattern language functions, namespaces, optimizations and new views

Documentation

A new documentation for ImHex (mainly the pattern language right now) is in the works and can be found here: https://imhex.werwolv.net/docs. It is a lot more detailed than the one found on the Wiki here and contains information about all the new features as well. Check it out!

Changelog

First of all, there has been made some adjustments to the distributed binaries:

• Windows releases are now distributed as an installer and a portable version
• Linux releases are now distributed as a flatpak file and a portable version
  • Thanks to @Calinou for the initial work on this
• MacOS support has been dropped because it's impossible for me currently to get it working correctly
  • The reason for this is that I have no access to a Mac and development is just frustrating for me
  • The previous release was already horribly broken because of that
  • If anybody would like to help fix ImHex on MacOS, PRs are highly appreciated.

For people that rather like to build ImHex themselves, there's now a Dockerfile present in the /dist folder of this repository

Additions

• Added a TTY console view (Windows only)
• CTRL + A now selects all bytes in the hex view
• Added default polynomials to CRC16 and CRC32 initially
• Added CPU usage to footer (Windows only)
• Added a new view for looking up common constants
• Added Create new File option
• Added Resize file option
• Added a UNIX permissions calculator tool
• Added a minimap to the data processor nodes editor
• Added crash detection and emergency backups
  • If ImHex crashes now, an automatic backup will be created and you will be asked if you want to restore it the next time you open ImHex.
• Got rid of the window border and title bar on Windows and instead use a custom one
• Added a toolbar with buttons
• Added proper interface scaling settings
• Added tips of the day popup at launch
  • This can be disabled from the popup or in the settings
• Added a tool to quickly upload files to https://anonfiles.com
• Added a tool to get short definitions of terms from Wikipedia
• Added half floats (16 bit floating point numbers) to data inspector
• Added UTF-8 strings to data inspector
• Added a built-in download "store" for patterns, libraries, magics and constant databases
  • The store directly accesses https://github.com/WerWolv/ImHex-Patterns so if you'd like to add more items to the store, please make a PR there

Pattern Language

• Added while statements for array definitions
• Added [[hidden]] attribute to hide a variable from the pattern data view
• Fixed the parent keyword not working properly
• Added ability to declare custom functions
• Allow fields in bitfields to be accessed like struct members
• Added support for namespaces
• Added support for declaring multiple variables on the same line
  • e.g u32 x, y, z;
• Moved all built-in functions to the std namespace
  • Most functions also got renamed to use snake_case instead
• Fixed endianess setting not applying to char16s
• Added full unicode support by default
  • ImHex now loads unifont on launch
• Optimize arrays of built-in types
  • This causes e.g huge byte arrays to not take up nearly as much memory anymore
  • The same optimization can be applied to custom structs by marking them with the new [[static]] attribute, however this only works correctly if the layout of the struct is actually static.

Improvements

• ImHex now reduces its framerate if there's no events to be processed and doesn't render at all anymore when minimized.
• Allow ImHex to redraw the screen when resizing for a much smoother experience
• Added a nice banner to the welcome screen
• Splash screen, banner and certificates are now embedded into the executable instead of needing to be in the right folder
• ImHex now makes sure that saved project files always have the .hexproj extension. Thanks to @jam1garner
• Added a bunch of icons to the welcome screen
• The splash screen is now DPI aware
• ImHex now more reliably detects when a file has been modified
• Floats in the data inspector are now displayed in decimal form if they are reasonably close to 0.0 and in scientific notation otherwise. Thanks to @jam1garner
• Drastically improved how bookmarks are displaying bytes
• Added buttons to calculator tool
• Reduced build times of ImHex quite a bit
• Massively improved string search memory usage
• ImHex now searches in the exeutable folder for plugins and resources again on Linux as well as in the system directories

Bug fixes

• Fixed advanced decoding separator in the hex view not respecting horizontal scrolling
• Fixed various issues related to plugin loading
• Fixed keyboard shortcuts not working
• ImHex no longer crashes on every exit

If you like my work, please consider supporting me on GitHub Sponsors, Patreon or PayPal. Thanks a lot!

Crash fixes and QoL changes

Changelog

Additions

• Added a list of all paths used by ImHex to the about page to allow locating them easily on platforms with non-default settings

Improvements

• Improve confusing "Quit application" popup text
• Allow using of system libraries during build instead of the bundled ones. Thanks a lot to @Mailaender
• Selecting the "Match Selection" checkbox now correctly updates the current region in the Hash and Disassembler view
• Display correct offsets in hex editor header when base address is not aligned to the number of columns
• The disassembler now always disassembles all selected bytes and inserts .byte "instructions" when a opcode couldn't be decoded.

Bug fixes

• Fix crashes when ImHex is installed in a privileged location (e.g C:/Program Files) and is not launched as Administrator
• Fix git commit hash and branch not being displayed properly in Release builds
• Fix broken header text in About popup
• Fix published name of ImHex to show as "Humanity" in Window's Program and Features window

If you like my work, please consider supporting me on GitHub Sponsors, Patreon or PayPal. Thanks a lot!

Splash screen, parent keyword, undo, redo and paste

Changelog

After many months I can finally release a new version. I'm really sorry for the delay but I'm currently working full time on my bachelor's thesis so I don't have much free time to work on ImHex right now. Hopefully in a few weeks again though :)

Additions

• Added a new and super fancy splash screen!
  • All resource loading will now be done while this screen is shown
• Added German localization. Thanks a lot to @tumGER for proof reading!
• Added Italian localization. Thanks a lot to @CrustySean!
• Updated ImHex icon with a new one
• Added FPS limiting
  • ImHex is now less power hungry and lowers its FPS to 5 when the window loses focus
• Added Paste, Undo and Redo function!
  • This has LONG been overdue
• Added saving and loading of data processor setups
• Added data size node to the data processor
• Added ability to copy values from the data inspector
• Data overlays now work everywhere
  • This means if the data processor changed the displayed data, this data will also show up in the pattern data view, inspector, analyzer, disassembler and every where else
• Added parent keyword to the pattern language
  • This allows you to access variables that are found in the parent of the current structure
• Added array index syntax to r-values in the pattern language
  • This also turned the addressof and sizeof built-in function into operators now so they no longer require ""
• Added UTF-16 character type and strings to the pattern language
• Allow nested use of types in the pattern language
• Added dataSize() function to the pattern language to get the currently loaded data size
• Added base_address pragma to the pattern language to automatically set the file's base address
• Loaded plugins are now displayed on the welcome screen
• Added slider to entropy graph to quickly skip through the file
• Added Recent Files to File menu
• Added ability to lock bookmarks so they can't be edited anymore until unlocked
• Added Font Awesome icons in a few places
• Use correct folder paths on Mac and Linux instead of looking for everything next to the executable
  • Check the README to know where things are supposed to go now!
• Added memory usage footer item on Windows
• Added buffer combine, slice and repeat nodes to the data processor
• Added currently loaded file name to the window title
• Added Close File option to the File Menu
• Added hex editor color highlighting opacity setting

Improvements

• Pattern language execution, disassembling, searching and many other things that may take a long time to complete are now run asynchronously
• Properly open some default views on first launch
• Greatly improved the Data Analyzer interface with ImPlot
• ImHex now uses the system file dialog instead of a custom one to open files
• Improved colors used on the welcome screen
• Variables placed out of bounds are now discarded instead of causing a error
• Improved goto command to now respect base addresses

Bug fixes

• Fixed multiple severe memory leaks in the pattern language
• Fixed bookmark names and comments refusing to be changed
• Fixed color attribute requiring a 0x prefix to understand the value
• Fixed region selection event only selecting first byte
• Fixed eval_depth default value being way too low
• Fixed ternary expressions in the pattern language not working everywhere
• Fixed bookmark name and comment not being loaded properly from a project file
• Fixed binary value in base converter being shifted by one
• Fixed pointers in the pattern language easily crashing ImHex
• Fixed MIME-based pattern loading popup being almost unusable. Thanks to @ThisALV for making me aware of this

If you like my work, please consider supporting me on GitHub Sponsors, Patreon or PayPal. Thanks a lot!

Light mode, Node data processor, custom encodings and more

Changelog

Additions

• Pattern language additions
  • Added new built-in functions
    • addressof : Gets the address of a variable
    • nextAfter : Gets the address directly after a variable
    • alignTo : Aligns a value to the provided boundaries
  • Added modulus operator %
  • Added dollar operator $. It acts as a value placeholder and returns the current address within the file
  • Added support for printing 128 bit values
  • Added attributes
    • The attribute syntax is similar to the one found in C++ [[attr("value")]] and allows configuring extra parameters for variables. Currently available are the following:
    • name : Overrides the name displayed in the pattern data view
    • color : Overrides the color displayed in the pattern data view
    • comment : Adds a comment to the variable in the pattern data view
• Added Light and Classic color mode, changeable in the new settings interface
• Added data processor
  • This is a huge feature that allows preprocessing of data before it is shown in the hex view through a node system.
  • Currently following features are supported
    • Bitwise operations
    • Boolean operations
    • Control flow (if and co)
    • Base64 and hexadecimal decoding
    • AES decryption (all commonly used modes)
  • Besides this there's also a Nodes API that allows plugin devs to implement their own nodes
• Added full localization support
  • Through custom plugins, ImHex can now be fully translated to other languages
  • Language selection is done through the settings menu
• Added custom file encoding features
  • This allows for example ROM hackers to provide so called Thingy files that map byte sequences to UTF-8 characters. These decodings will be displayed to the right of the ASCII pane in the hex view
• Added set base address option to hex editor
• Added settings interface
• Added color picker for Bookmark header and selection color
  • Also added a tooltip window for bookmarks when hovering over their region in the hex view
• Added keyboard navigation. Thanks to @csinkers
• Improved suggested pattern popup to now list all available patterns instead of just the first one
• Added a nice welcome screen
• Added support for Home, End, PageUp and PageDown keys in hex view
• Added base converter tool
• Added footer and an API to add information to it

Improvements

• Most popup windows can now be closed with the ESC key
• All built-in features of ImHex are now refactored into a "builtin" plugin
• Improved about page to contain actual links and better information
• Improved error displaying for when files failed to open
• Switched to mbedtls instead of openssl for crypto operations

Bug fixes

• Fixed issues with plugins
• Fixed signed integer display in data inspector
• Fixed many crashes especially on Linux
• Disabled multi-viewport support on Linux as it's just not ready yet to be used there
  • Once ImGui updates and adds better support for Linux, this will be reverted
• Fixed multiple memory leaks
• Fixed search feature in hex view
• Fixed hex view shortcuts
• Fixed multiple instances of popups only appearing when certain views are open
• Fixed bookmarks being created and displayed in wrong positions in huge files

If you like my work, please consider supporting me on GitHub Sponsors, Patreon or PayPal. Thanks a lot!

Windows Installer and bug fixes

Changelog

Additions

• Added a MSI installer for Windows
  • Thanks a lot to @roblabla for his work on this!
• Added a console to the pattern window to show evaluator infos, warnings and errors
• Added assert, warnAssert and print builtin functions
• Added a right click context menu to the Hex editor window
• Added engineering display mode to Calculator
• Various UI improvements

Bug fixes

• Fixed findSequence builtin function not working correctly
  • Thanks to @kaero
• Fixed issues with bitfields, enums and unions
• Fixed a bunch of crashes when using the pattern language
• Fixed invalid display of 64 bit values in the pattern data window
• Fixed bitfield entries displaying an invalid type
• Fixed empty structs displaying a wrong offset
• Fixed a off-by-one error that prevented arrays from going to the end of the data
• Fixed boolean OR operator not working at all
• Fixed applying of defines
• Improved UTF-8 display in data inspector
• Various cmake improvements

If you like my work, please consider supporting me on GitHub Sponsors, Patreon or PayPal. Thanks a lot!

Pattern Language rewrite, better file IO performance and macOS support

Changelog

Additions

• Added official support for MacOS
  • Made possible by @Thog's amazing contributions to the build system
• Added Plugin support
  • Allows creation of custom windows, custom entries to the Tools window and custom data providers
  • An example can be found under /plugins/example
• Rewrote the entire pattern language parser and evaluator
  • The language is now faster, more robust and has much less issues
  • A in-depth guide for all the features in the Pattern Language can now be found in the Wiki
• Many new features have been added to the pattern language
  • Added mathematical expressions with many operators, bit operators and value access
  • Added a much more featureful way of using values of other variables everywhere where a number is required
  • Added conditional statements for structs and unions (if, else and else if)
  • Added back zero sized array. These will simply be ignored
  • Added scope resolution operator for using enum values in expressions
  • Added built-in functions to find sequences and read values at addresses
  • Added boolean patterns
  • Added unsized character arrays for null-terminated strings
• Use file memory mapping instead of using standard file IO
  • This greatly increases performance on older computers and brings down background file IO to a minimum.
• Added Decimal, Hexadecimal and Octal number format options to data inspector
• Added nightlies for all platforms
  • These are available under the Actions tab
  • Thanks to @Thog for the macOS nightlies
• Added full DPI scaling
• Added custom font support
  • To use this, place your font named font.ttf next to the executable
• Added version info to about page
  • Version info, current commit hash and branch are shown
• Added selected region information to hex editor footer
• Various UI/UX improvements

Bug fixes

• Greatly improved byte highlighting performance
  • Previously, when scrolling down with a big pattern running, huge slowdowns started happening. The algorithm used for highlighting has been greatly improved to fix this.
• Fixed opening of files with a non-ASCII file path
• Fixed crashes with zero-sized arrays
• Fixed byte selection in read-only files
• Fixed ImHex refusing to be closed in some circumstances
• Fixed data inspector not updating in all cases
• Fixed ASCII characters drawing above the hex editor footer
• Fixed floating point displaying in pattern data view
• Fixed pattern data not updating when a new file is loaded

Python loader scripts, individual variable endianess and math evaluator

Changelog

Additions

• Added math evaluator / calculator to tool window
• Added overriding of endianess of individual variables instead of the entire file
• Added Python loader scripts. Right now you need to have some version of Python 3.8.X installed, otherwise rename the lib/python3.8 folder to lib/python3.9 (or whatever your version is)
  • This is very bare-bones right now but more will come in the future. Features so far are:
  • Applying patches
  • Creating Struct and union definitions
  • Getting of currently loaded file path
• Added cursor movement with arrow keys in hex editor
• Added patches list window
• Added feature to apply IPS and IPS32 patches
• Added simple project files
• Added opening files by dropping them on the executable (this also means opening files from the command line)

Bug fixes

• Fixed opening files not working when hex editor view isn't open
• Fixed shortcuts from applying to closed windows
• Fixed help view appearing in View menu
• Fixed crash when array size variable read the value zero
• Fixed data inspector failing to read values at the end of the file
• Changed cheat sheets from modal window to actual window to allow keeping it open
• Prevented console from opening on Windows
• Fixed possible crash when loading files
• Fixed user not being asked if a pattern file should be automatically loaded, even if there was one available

Thanks a lot to @Thog for her incredible work fixing up the dependency mess and implementing CI for both windows and Ubuntu :)

IPS patches, error messages and UI cleanup

Changelog

Additions

• Added IPS and IPS32 exporting of patches
• Added proper error messages and error highlighting to the pattern language window
• Added import from Base64 option
• Added copy from string and copy demangled string to strings window
• Added support for demangling both Itanium and MSVC symbols
• Added Save and Save As button to save modifications to disk
• Added character literals to pattern language
• Added simple bookmark and comment feature. No way to save them yet though
• Make types and disassembly mnemonics be colored
• Clicking on pattern variables, strings and disassembly table rows now highlights the relevant bytes
• Massively improved look and feel of many different windows
• Enabled ImGui's Viewport support. This means you can now detach windows and move them to a different screen
• Improved ASCII and Wide character display in inspector

Bug fixes

• Fixed windows being tiny at first start
• Removed collapse button from all windows since it doesn't make sense
• Fixed string search yielding invalid results in the last searched block
• In the pattern data view, the variable's name is now indented instead of the color
• Shrink color picker in the tools window a bit since it was really huge

Bug fixes

Changelog

Bug fixes

• Fixed a crash when no patterns folder exists
• Fixed a crash when scrolling the disassembler options child off screen
• Fixed time_t decoding crash on Linux
• Truncate file size to show only two decimal places
• Fixed format function crash when providing an invalid pattern