The need within EATS for technical tooling to convert Java code to PHP is not a one time need. There is no intention to abandon Java as a language for code development for EATS. But there is a need for consistency of logic and operation between cyber processes which are specified in Java, and other cyber processes which are specified in PHP, where both sets of processes are part of a unified cyber architecture. What is not needed, but rather is contraindicated, is a requirement for manual translation of machine function specification on an ongoing basis as new Java code is developed, performing functions which are desired to be exposed in both realms of implementation; or the manual translation of adjustments as functions are corrected, and otherwise evolved, over time. The translator, and translation facility, needs to become a functional, maintained, component with the system architecture.
The historical pattern for this relates back to lessons learned by the author from:
- Philip W. Metzger’s Managing a Programming Project (1973)
- Various project documents related to software development efforts in support of Rome Air Defense Command (RADC) which was published by the National Technical Information Service (NTIS) in the 1970’s and 1980’s.
- The Programming Support Library (PSL) concept conceived by Harlan Mills as part of his exposition of Chief Programmer Teams in IBM’s 1971 development of the New York Times project.
- The use of Copylibs, and the development of a Copylib cross-reference tools for code management of the Securities Inventory Control System through the 1970’s and 1980’s.
- The development of SysModel as an extension to the What Used/Where Used facility provided by IBM as part of their Consumer Transaction Facility (CTF) as used by Bank of America in the 1990’s.
JavaToPhp is an open source project by Devjino which was last updated in 2012 when it was identified and first used for EATSv5 in April 2020. It consisted of a single Java program, Compiler.java, and an incorporated version of the Java Parser, incorporated as a JAR library. It was described as “a Java source code ‘compiler’ with PHP source code output.” The program uses two lines of code (a file read, and a call to the parser library) to read and parse a Java source program. The result of the parsing is an Abstract Syntax Tree (AST) representing a structured form of the logic specified by the source program. JavaToPHP then proceeds to parse the AST and output a PHP translation of the Java program.
In the code as obtained from GitHub, the parser, while old, is reasonably complete; and the new version includes documentation on the parser itself. The compiler, on the other hand, is neither complete, nor documented. But it is functional.
- new is used to create Primitives and byte arrays
- old code used embedded classes for things where new code uses lambda expressions, both should have a common fixup
As provided the compiler treated all errors as fatal, throwing a hard exception when an error was encountered. In one context this makes sense. For our use, it doesn’t. Throwing hard errors forces the one by one resolution of problems in a situation where the problem is missing functionality in the compiler rather than any error in the code being translated. Instead, the code was modified to report errors and keep processing in order to determine as many issues as possible with each run of the program.
Java Parser Upgrade
The JavaToPHP source used a javaparser-1.0.10 JAR library. The newest available library is javaparser-core-3.3.0.jar. An attempt to migrate directly to the 3.3 library resulted in a series of errors in the compiler code, some of which are warned about in the most recent upgrade documents. (Examples include migration of functions between packages.)
A first step migration to javaparser-core.2.1.0.jar avoided the most immediately concerning of these errors, and resulted only in errors associated with functions which were not required to support the original code. This was used as a half-way, stepping stone, migration point to migrate the code forward in a piece-meal basis. The primary problem with the v2 parser related to the handling of comments, which was not working with the v1 parser either, so it made the upgrade issue moot. At the same time, a partial benefit of the move to v2 was the exposure of an error related to lambda expressions in the parser which was a sign of the forward movement of the parser relative to new features in the language.
After exposing and resolving a number of issues in the compiler at the parser v2 level, we moved to v3.This generated a slew of errors in the JavaToPHP functional code, at least some due to realignment of the functions moving between parser packages, and types being changed which caused cascading problems. (Example: Comment as a type became Optional<Comment> using generics.) Absorbing these changes can take time, lots of time. Seeing these types of things emerge is why I have a major hesitancy to rely on externally supplied libraries1This is also why EATS exists as a middleware layer. Direct use of library code, where library authors feel free to deprecate their API rather than support it is a significant cause of the software turmoil in the current age. It needs to be eliminated, to the extent possible, in order to be able to truly move to a point where cyber-physical systems can be trusted, or the alternative is a deep dark pit when an update to a life support system fails because of a minor update, by a “got a better idea how to do this” software engineer, kills one or more people. The main trouble with a lot of software (and related pseudo-engineered products) is people, those who engineer , construct, and maintain it, and the managers who push them, and an apparent mutual disdain for quality over quantity and time-to-market. Come hell or high water. Users of cyber need protection from these people.. After an attempt to bite the bullet and take the time to upgrade, with a jar file that contained no JavaDoc, I found playing with v3 to not be worth the effort. This was reinforced when I found the number of un-resolved bug reports on the code.
That’s why we backed off and stayed on v2, but moved forward to a v2.5 archive from the v2.1 version we had been using.
- AF_EATSv5 PhpCompiler JavaDoc is the current JavaDoc for our modified Java To PHP compiler.
- JavaParser Home is the web site for the JavaParser used to construct the AST.
- Visitor Pattern Explained contains a description of the Visitor Pattern which is the the conceptual pattern for use of the JavaParser.
- The free book: JavaParser Visited is the external, beyond the web site and Java, documentation for JavaParser.
- JavaDoc javaparser-core 2.5.1 is the JavaDoc for v2.5.1, which is the version of JavaParser currently in use by our code.
- JavaDoc javaparser-core 3.2.5 is the JavaDoc for v3.2.5, which is the current version of JavaParser.
AST Component Extensions
The first series of functional modifications to the code included extension of the AST node types processed. An understanding of missing nodes, and how to handle them is typically available from the Java documentation in the parser library, supplemented by the external documentation for the parser.
File Handling Changes
As provided the compiler was passed a command line input parameter describing the file(s) to be processed, and results were written to STDOUT, and error messages to STDERR, such that the entire process was scripted for potential use as a pipeline process. Multiple input files could be supplied, with a single output stream created collecting all of the generated PHP code.
This was changed to a framework of use of the compiler as a subroutine process designed for the translation of one input program on each execution of the compiler, and a multiple program conversion capability was implemented by cascading the command line execution to a two tier process, where a top tier provides a set of repeated calls to a generic shell (command line) script which implements the actual call for each program. In this manner, a single execution can still be used to translate an entire library of Java programs, but each program is provided individual input and output file management, including where to source inputs and where to send outputs. This allows commonly structured directories to be maintained for Java and PHP code, with the compiler translation sourcing from the Java collection, and producing output in a separate PHP collection, with automatic synchronization between the two.
Comment Carry Forward
Added a feature to carry forward the comments from the Java code to the PHP code as an aid in understanding of the PHP code when debugging, and when needing to manually adjust the translated code.
Added a feature to gather cross reference information during the code translation and output the results as a CSV file. Content includes the source module where the content was derived, the type of relationship between the source and the referenced data, and a data reference to an element which was imported, a class used as a type, classes that are instantiated, etc. The direct reading of the file is a form of what used reference, sorting by element, it becomes a where used reference. The file is concatenated for each module translated. The where used summary defines the class and import set required to be supported in the PHP environment, and thus becomes a source of specification for Java classes which need to be supported as classes in PHP for the translated code to operate without further manual adjustment.