Highlight Automated Code Scan (Command Line)
This page details how to automate source code scans by leveraging the Highlight command line and integrating it within your build or CI/CD environments. This will allow you to continuously track and monitor software health and cloud readiness of your projects and applications. For any questions not addressed in this documentation, don’t hesitate to contact our support team.
Before getting started…
What is a Command Line?
Please read and accept the license agreement prior to downloading the command line.
How to use the command line
Examples
Find below some examples of options you can reuse for your own code scan configuration.
java -jar HighlightAutomation.jar --workingDir "C:\highlight-myproject" --sourceDir "C:\myproject\src" --skipUpload
java -jar HighlightAutomation.jar --workingDir "/home/user/highlight-myproject/" --sourceDir "/home/user/svn/myproject/src/" --skipUpload
java -jar HighlightAutomation.jar –propertiesPath “C:\tmp\highlight.properties”
java -jar HighlightAutomation.jar --workingDir "C:\highlight-myproject" --sourceDir "C:\myproject\src" --technologies "Java,Python" --skipUpload
java -jar HighlightAutomation.jar --workingDir "C:\highlight-myproject" --sourceDir "C:\myproject\src" --ignoreDirectories "test,jquery" --skipUpload
java -jar HighlightAutomation.jar --sourceDir "C:\folder1" --sourceDir "C:\folder2" --workingDir "C:\temp" --skipUpload
java -jar HighlightAutomation.jar --workingDir "C:\highlight-myproject" --sourceDir "C:\myproject\src" --ignorePaths ".*\/vendor\/js" --skipUpload
java -jar HighlightAutomation.jar --workingDir "C:\highlight-myproject" --sourceDir "C:\myproject\src" --ignorePaths ".*test.*" --skipUpload
java -jar HighlightAutomation.jar --workingDir "C:\highlight-myproject" --sourceDir "C:\myproject\src" --ignorePaths "./test.*" --skipUpload
java -jar HighlightAutomation.jar --workingDir "C:\highlight-myproject" --sourceDir "C:\myproject\src" --login "john.doe@acme.com" --password "*******" --applicationId 1234 --companyId 5678 --serverUrl "https://rpa.casthighlight.com"
java -jar HighlightAutomation.jar --workingDir "C:\highlight-myproject" --sourceDir "C:\myproject\src" --ignoreFiles ".*foo.*" --skipUpload
java -jar HighlightAutomation.jar --workingDir "C:\highlight-myproject" --sourceDir "C:\myproject\src" --ignoreFiles ".*foo.*\.js" --skipUpload
java -jar HighlightAutomation.jar --workingDir "C:\highlight-myproject" --sourceDir "C:\myproject\src" --ignoreFiles ".*\.vue" --skipUpload
java -jar HighlightAutomation.jar --uploadZipFile "C:\scans\HighlightResult.03_30_2022_14_40.zip" --serverUrl https://rpa.casthighlight.com --companyId 1234 --applicationId 5678 --tokenAuth *****-****-****-****-******* --workingDir "C:\temp"
java -jar HighlightAutomation.jar --dockerImageNameTag node#latest --includeArchiveContent=3 --workingDir "C:\temp" --skipUpload
Command Line Options
--help Displays the different options
--printTechnos Print the supported technologies (e.g. Java, Python, COBOL, etc.)
Scan Options
--propertiesPath (optional) The absolute path to the .properties file that will centralize and drive all other options of the command line.
--sourceDir (mandatory) The absolute path to the directory that contains the source code to be scanned by CAST Highlight.
--workingDir (mandatory) This is the absolute path to the Highlight working directory. Within this directory, a Highlight temporary folder ("HLTemporary") will be created and will contain scan result files (CSVs located in HLTemporary/analysis folder) and analysis log files (CSVs located in HLTemporary/discover folder). To make it short, this is the directory where you want to store scan results.
--technologies (optional) Technologies you want to explicitly scan in your sources. Separated by "," (See --printTechnos option above).
--ignoreDirectories (optional) Directory name patterns to ignore during the scan (e.g. test folders, .git, etc.). Separated by ",". Source code within directories matching with these patterns will be automatically excluded from the scan.
--ignorePaths (optional) List of regular expressions to ignore paths, separated by "|". Source code within directories matching with this regexp will be automatically excluded from the scan. Note that starting ^ and ending $ are implicit and don't need to be passed in this option. --ignoreFiles (optional) List of regular expression to ignore file names. Separated by ",". Files matching with these patterns will be automatically excluded from the scan. Example to exclude all files containing "foo" with a .js extension: --ignoreFiles ".*foo.*\.js"
--analyzerDir (optional) Alternate directory for Highlight's analyzer scripts.
--perlInstallDir (optional) Directory of perl installation (default: C:\Users\{user}\AppData\Local\CAST Highlight Code Reader\strawberry\perl).
--keywordScan (optional) Path and filename of your KeywordScan XML configuration file (e.g. C:\temp\KeywordScanner_GDPR.xml). Read this post for more information on the feature.
--analyzeBigFiles (optional) Will bypass the file size limitation of the analyzers (will eventually take longer to scan).
--allowGeneratedCode (optional) Will bypass the default exclusion of generated code files.
--mavenRepository (optional) Path to local .m2 repository for better detection and version resolution of Maven-based dependencies.
--includeAllDependencies (optional) Will bypass the automatic filtering of indirect dependencies found in node_modules.
--includeArchiveContent={levelOfDepth} (optional) Will extract content of archives (.jar, .tar, .zip, .war files) and add it as part of the scan. {levelOfDepth} is the level of depth within archives you want to apply (e.g., "includeArchiveContent=2" will recursively look into 2 level of archive files). Note that this option can increase the scanning time. Although it is not recommended, you can use higher level of depth (max 99). Finally, note that scan of password-encrypted archives or symbolic link is not supported.
--dockerImageNameTage {imageName#tag} Will scan the Docker image corresponding to the provided tag (use '#' as a separator between image and tag). See this article for more information.--dockerImageNameTage {imageName#tag} Will scan the Docker image corresponding to the provided tag (use '#' as a separator between image and tag). See this article for more information.
--dbgMatchPatternDetail Will produce detailed files for Cloud/Green insights for our Visual Studio Code plugins
Upload Options
Below are the required options to use when you want to automatically upload scan results to the Highlight platform. Then, the option “–skipUpload” should be removed.
--login Login of an active Highlight user.
--password Password for the login indicated above.
--basicAuth Alternatively to login/password, you can use this option to pass your credentials encoded in base64
--tokenAuth (recommended) Alternatively to Basic authentication, you can use this option to pass your OAuth token --companyId (mandatory) Identifyer for the company (can be retrieved from the Highlight portal, it is the ID displayed in the url when clicking on the top-level domain in "MANAGE PORTFOLIO > MANAGE APPLICATIONS" from the menu).
--applicationId (mandatory) Identifyer for the application (can be retried from the Highlight portal, it is the ID displayed in the url when editing an application in "MANAGE PORTFOLIO > MANAGE APPLICATIONS").
--serverUrl (mandatory) The Highlight server instance where the results has to be uploaded (user credentials have to work on this server). E.g. 'https://rpa.casthighlight.com'
--snapshotDatetime (optional) Time (epoch) to use for uploaded application snapshot.
--snapshotLabel (optional) The application snapshot label you want to display on the application result page on the portal (e.g. release version, build number, etc.).
--skipUpload (optional) Will generate CSV results only, no result upload will be performed.
--appendResult(optional) This option will add uploaded result CSVs to existing/previously uploaded CSV results. This option should be combined with --skipSubmit if you expect further CSV result uploads from other scans for the same application.
--skipSubmit (optional) CSV results will be uploaded but this won't trigger the result processing (score calculations, component detection, etc.) by the CAST Highlight platform.
--zipResult (optional) This option will create a ZIP file containing CSV result files (e.g., --zipResult C:\\temp\\foo.zip).
--uploadZipFile (optional) This option will upload a ZIP file containing result CSVs without running a code scan (e.g., --uploadZipFile C:\temp\foo.zip). Note that the --workingDir option must be present.
Log files
The log file (HLAutomation.log) is produced after the command line is run and is stored in the working directory (–workingDir) that has been set in the options.
If you perform operation in your build environment (prefered solution)
You may directly add –mavenRepository <path_to_m2_folder> to indicate the local maven repository location and ensure build as already been done.
Otherwise, you’ll have to install Maven on your machine and have access to maven external repositories required for your application.
- Download and install Maven (https://maven.apache.org/install.html)
- From the source folder where the root pom.xml is, run the following Maven command which will calculate the exact dependencies and versions of the project:
$> mvn clean install
$> mvn org.apache.maven.plugins:maven-dependency-plugin:3.8.0:tree
- Ensure that maven m2 repository is available and can be read. Windows : default configuration is to have repository in C:\Users\<username>\.m2. Check the visibility status (hidden option in windows explorer). Linux/Mac: check owner and right available on directory, should be at least “read”.
- Scan the source folder with the Command Line with –mavenRepository <path_to_m2_folder>, optionally with the option –includeAllDependencies if you want to get the whole dependency tree
- To ensure that option is correctly working check data in framework.validated.csv, with option activated more components should be reported with a version defined.
Example
$> java -jar HighlightAutomation.jar --sourceDir "C:\source" --workingDir "C\source\hlresults" --mavenRepository "C:\Users\<username>\.m2" --skipUpload
Proxy with no password
-Dhttps.proxyHost=<your proxy host>
-Dhttps.proxyPort=<your proxy port>
Proxy with credentials:
If a password is required, add the following additional parameter
-Dhttps.proxyUser=<user>
-Dhttps.proxyPassword=<password>
Example
java -Dhttps.proxyHost=your proxy host -Dhttps.proxyPort=your proxy port -Dhttps.proxyUser=user -Dhttps.proxyPassword=password -jar HighlightAutomation.jar --workingDir "C:\highlight-myproject" --sourceDir "C:\myproject\src" --login xxx --password xxxx
Requirements
Java 11 and above is required.
Prior using the command line, it is preferable to install CAST Highlight’s Code Reader on your machine as it embeds the required Perl 5.36.3.1 binaries. Alternatively, you can install the required Perl distribution (Strawberry MSI Installer) and modules (see below) if you don’t want to install the Code Reader. The Code Reader can be downloaded from your Highlight user account, in the APPLICATION SCANS page.
- Digest::SHA
- XML::LibXML
- JSON
- Time::HiRes
- Math::BigInt
- Win32::LongPath
Example of CPAN commands to install these Perl Modules:
- cpan install Digest::SHA
- cpan install XML::LibXML
- cpan install JSON
- cpan install Time::HiRes
- cpan install Math::BigInt
- cpan install Win32::LongPath
For Linux Debian/Ubuntu
The following components must be installed on your machine: Perl 5 (tested on Strawberry 5.36.3.1), libjson-perl, libxml-libxml-perl
$>perl -v
on debian/ubuntu systems
$>dpkg –get-selections libxml-libxml-perl libjson-perl
Java 11 and above
$>java -version
For Linux – RHEL/CENTOS based systems
Install the libraries: perl-XML-LibXML, perl-JSON and perl-Digest-SHA
$>yum -y install perl-Digest-SHA
$>yum -y install perl-JSON
$>yum -y install perl-XML-LibXML
$>yum -y install perl-Time-HiRes
$>yum -y install perl-Math-BigInt
For MacOS
For MacOS users, using our Docker image is recommended to run the command line as a container.
$>perl -v
$>cpan install XML::LibXML
$>cpan install JSON
For SSO/SAML users
SSO/SAML users must use user tokens to authenticate and upload result files to the CAST Highlight platform. See how the feature works.
0 – Ok
1 – Command Line general failure
2 – Command Line options parse error
3 – Command Line techno discovery error
4 – Command Line analysis error
5 – Command Line result upload error
6 – Command Line source dir or output dir validation error
7 – Command Line result saving to zip file error
8 – Command Line upload from zip file error
CLI logs
Running the command line will create an HLAutomation.log that contains traces of a scan. This file is located in the specificed working directory (–workingDir).
Generated Code exclusion
By default, the command line excludes files that are known as generated code. These files are following the patterns below. In case you would want to scan these files, use the –allowGeneratedCode option.
*.designer.vb, *.designer.cs, *.reference.cs, *.reference.vb