Command Line Operation

Batch operation is possible with installed applications and from Python source code.
Run as follows:
  • Python source code:
python arelleCmdLine.py arguments
  • Windows x64 or 32 bit:
"c:\Program Files\arelleCmdLine.exe" arguments
  • MacOS App:
/Applications/Arelle.app/contents/MacOS/arelleCmdLine arguments
  • Linux/Unix:
./arelleCmdLine arguments
(for Sparc/Solaris or otherwise as needed may need to set .so  loading path:
LD_LIBRARY_PATH=`pwd` ./arelleCmdLine arguments  ...)
where arguments are:
  • -h or –help for the arguments supported by the running version of the program
  • -a or –about for the compile date version, copyright, and license information
Specifying input files
  • -f or –file followed by file name (entry point to be loaded, instance, schema, linkbase, inline XBRL instance, testcase file, or testcase index file).  File may be local or a URL to a web-located file.  (If packages are loaded and enabled, remapping of URLs into package files is effective.) File may be a JSON structure of one or more files and applicable parameters based on plugin consumption: –file ‘[{“file”: “foo.xml”},{“file”:”bar.xml”}]’
  • –username followed by username if needed (with password) for web file retrieval
  • –password if needed (with username) for web file retrieval
  • -i or –import followed by a list of files to import to the DTS, such as additional formula or label linkbases.  Multiple file names are separated by the “|” character.
  • -d or –diff followed by file name for a second DTS to be compared for versioning report generation.  In this case the -f file is the fromDTS and the -d file is the toDTS.
    • -r or –report, followed by the file name into which to save a versioning report (companion to -d or –diff).
Validations
  • -v or –validate: perform validation according to the file type.  If an XBRL file, it is validated according to XBRL validation 2.1, calculation linkbase validation if either –calcDecimals or –calcPrecision are specified, and disclosure system validation (as selected). If a test suite or testcase, the test case variations are individually so validated.  If formulae are present they will be validated and run unless –formula=none is specified.
  • –calcDecimals: select calculation linkbase validation, inferring decimals (current recommendation)
    • –calcPrecision: select calculation linkbase validation, inferring precision instead of decimal (deprecated, prior XBRL 2.1 recommendation)
  • –disclosureSystem followed by system code (such as ‘efm’), for disclosure system rules validation.  Enter –disclosureSystem=help for a list of names, or help-verbose for a list of names and descriptions.  (Replaces –efm and –gfm, which are deprecated.)
  • –utr: select validation with respect to the Unit Types Registry.  (Some disclosure systems validations activate utr validation or provide specific utr files, so this entry would not be needed, such as U.S. SEC.)
    • –utrUrl followed by URL overrides the systems Unit Type Registry (with a URL or local file path) (not needed with a –disclosureSystem argument).
  • –infoset specifies validation with respect to test case infosets (which are part of some test case suites).
  • –labelLang provides an xml:lang language code to override the system language settings for labels output by the following file options, e.g., –labelLang=en-US
  • –labelRole overrides the standard XBRL 2.1 label role for the labels output by the following file options, e.g., –labelRole=http://www.xbrl.org/2003/role/terseLabel
The following file options determine the type of file saved by the extension of the file name, the extension may be any of: “.html”, “.htm”, “.xhtml”, “.json” or “.csv”.  The prior options named –csv* are deprecated but still work.
  • –DTS followed by file name: write the DTS tree into the specified file
  • –facts followed by file name: write the fact list into the specified file
    • –factListCols followed by a list of columns for a –facts list, the columns are comma separated and can be any of these: Label, Name, contextRef, unitRef, Dec, Prec, Lang, Value, EntityScheme, EntityIdentifier, Period, or Dimensions. The default columns list is “Label contextRef unitRef Dec Prec Lang Value“.
  • –factTable followed by file name: write the fact table into the specified file
  • –concepts followed by file name: write the concepts (elements) list into the specified file
  • –pre followed by file name: store the presentation linkbase in specified file
  • –cal followed by file name: store the calculation linkbase in specified file
  • –dim followed by file name: store the dimensions in definition linkbase in specified file
  • –anch followed by file name: store anchoring relationships in specified file
    • The wider-narrower hierarchy is shown in a tree view.
    • For each anchored concept, if there are multiple parent (wider) concepts, the others are shown in a second column, “wider”.
    • To additionally show name, documentation and references on this view, specify one or all of –relationshipCols “Name,Documentation,References”
    • –formulae followed by file name: store the formula-related resources tree in specified file
  • –viewArcrole followed by an arcrole: store relationships for the arcrole in the file specified by –viewFile
    • –viewFile specifies the file to store –viewArcrole relationships
  • –roleTypes followed by file name: store defined role types in specified file
  • –arcroleTypes followed by file name: store defined arcrole types in specified file
  • –testReport followed by file name: write a test report of validation (of test cases) in specified file
    • –testReportCols followed by a list of columns for a –testReport, the columns are comma separated and can be any of these: Index, Testcase, ID, Name, Reference, ReadMeFirst, Status, Expected, Actual“.
  • –rssReport followed by file name: write a test report of RSS feed processing in specified file
    • –rssReportCols followed by a list of columns for a –testReport, the columns are comma separated and can be any of these: Company Name, Accession Number, Form, Filing Date, CIK, Status, Period, Yr End, Results”.
Logging options:
  • —logFile: write log messages into file, otherwise they go to standard output. If file ends in .xml it is xml-formatted, otherwise it is text.  XML log files contain significant additional details not provided in the text files.  The file names logToPrint, logToStdOut.xml and logToStdError provide output on standard output and standard error streams.
    • –logFormat: specify the Python logger format for the text description of messages, if absent the default is [%(messageCode)s] %(message)s – %(file)s.
    • –logLevel: Minimum level for messages capture, otherwise the message is ignored.  Current order of levels are debug, info, info-semantic, warning, warning-semantic, warning, assertion-satisfied, inconsistency, error-semantic, assertion-not-satisfied, and error.
    • –logLevelFilter: Regular expression filter for logLevel. (E.g., to not match *-semantic levels, –logLevelFilter=(?!^.*-semantic$)(.+).
    • –logCodeFilter: Regular expression filter for log message code
  • –collectProfileStats: Collect profile statistics, such as timing of validation activities and formulae.
Validation and formula options:
  • –parameters specifies parameters for formula and validation (name=value[,name=value]).
    • –parameterSeparator: Specify parameters separator string (if other than comma, may be a multi-character separator).
  • –formula: Specify formula action, If this option is not specified, -v or –validate will validate and run formulas if present:
    • validate – validate only, without running;
    • run – validate and run, or
    • none – prevent formula validation or running when also specifying -v or –validate.
  • –formulaParamExprResult: formula tracing option
  • –formulaParamInputValue: formula tracing option
  • –formulaCallExprSource: formula tracing option
  • –formulaCallExprCode: formula tracing option
  • –formulaCallExprEval: formula tracing option
  • –formulaCallExprResult: formula tracing option
  • –formulaVarSetExprEval: formula tracing option
  • –formulaVarSetExprResult: formula tracing option
  • –formulaVarSetTiming: show times of variable set evaluation
  • –formulaAsserResultCounts: formula tracing option
  • –formulaFormulaRules: formula tracing option
  • –formulaVarsOrder: formula tracing option
  • –formulaVarExpressionSource: formula tracing option
  • –formulaVarExpressionCode: formula tracing option
  • –formulaVarExpressionEvaluation: formula tracing option
  • –formulaVarExpressionResult: formula tracing option
  • –formulaVarFiltersResult: formula tracing option
  • –formulaVarFiltersWinnowing: shows winnowing process of successive application of formulas, helpful in debugging to find out which filter affects the facts that pass a filtering step
  • –formulaVarFiltersResult: shows result of filters
Configuration options
  • –uiLang: Language for user interface (override system settings, such as program messages).  Does not save setting.  Does not affect labels language of above reports (see –labelLang).
  • –proxy: Modify and re-save proxy settings configuration. Enter system to use system proxy setting, none to use no proxy, a specific prlxy in the format http://[user[:password]@]host[:port] (e.g., http://192.168.1.253, http://example.com:8080, http://joe:secret@example.com:8080), or show to show current (saved)setting.
  • –internetConnectivity: Specify internet connectivity: online or offline.
  • –internetTimeout: Specify internet connection timeout in seconds (0 (zero) means unlimited).
  • –internetRecheck:  Specify frequency of rechecking web file’s time stamps to cached file copy and determine if newer file to be reloaded: weekly (default), daily or never.
  • –noCertificateCheck: Skip https SSL certificate checking.
  • –internetLogDownloads:  Log an info message for downloads to the web cache.
  • –xdgConfigHome: Specify non-standard location for configuration and cache files (overrides environment parameter XDG_CONFIG_HOME, if also provided).
  • –plugin[s]: specify plugins to load (no longer saved or shared with GUI):
    • Module urls are ‘|’ separated. Standard plugin modules do not need “.py” specified.
      • relative URLs are relative to installation plug-in directory
    • Packaged plug-in urls are their directory’s url.
  • –packages:  Modify taxonomy packages configuration. (not saved or shared with GUI)
    • Package urls are ‘|’ separated:
    • URLs are full absolute paths.
  • –abortOnMajorError: Abort process on major error, such as when load is unable to find an entry or discovered file.
  • –webserver: Start web server on host:port[:server] for REST and web access, e.g., –webserver locahost:8080, or specify nondefault a server name, such as cherrypy, –webserver locahost:8080:cherrypy.
  • (It is possible to specify options to be defaults for the web server, such as disclosureSystem and validations, but not including file names.)

Plug in Options

Many of the plug-ins add command line (in addition to GUI menu) options.  For additional detail see the plug-in documentation.  A summary of standard plug options is provided here:

inlineDocumentSet

Specify an inline XBRL document set by JSON parameter: –file ‘[{“ixDocumentSet”:[{“file”:file1},{“file”:file2}…]}]’
–save-instance: provides a file name into which to save an XBRL instance document derived from a loaded manifest (Japan FSA) of a document set of inline documents, or loaded single inline document.  (Currently supports one target instance document only.)

loadFromExcel

(no command line options, but the -f and –file options can specify an Excel document which represents a DTS.)

profileCmdLine

–save-profiler-report: specifies file into which to save a Python profiler report of the timings of Arelle Python methods and routines for the operations performed by the other command line functions.  Useful to determine if a long running function is spending excessive time in a function that needs implementation attention.

saveDTS (produces TaxonomyPackage)

–package-DTS: specifies a file into which to save a Taxonomy Package representing the loaded DTS

saveHtmlEBAtables (EBA Table Sets)

–save-EBA-tablesets: specifies an index file (usually index.html) to represent an html page that holds a table of contents to the EBA tables generated for all loaded EBA Table Linkbase tables in the DTS.

saveLoadableExcel

–save-loadable-excel:  Saves an Excel workbook representing the DTS (which can be subsequently loaded by loadFromExcel.py plugin.

saveSKOS

–save-skos: saves a SKOS (OMG standard OWL file) representing a semantic view of a DTS based on presentation linkbase entries (for US-GAAP, IFRS, and EDInet style taxonomies).

validateSecTagging

–save-sec-tag-dts-matches: adds to disclosure system SEC validation checks for SEC non negatives based on an algorithmic and weighted approach
sphinx (subpackage, features including generating formula linkbase have not been sufficiently tested)
–import-sphinx: Import sphinx files to the DTS for validation.  Multiple file names are separated by a ‘|’ character.
–generate-sphinx-formula-linkbase:  Generate an XBRL formula linkbase from sphinx files.  Multiple file names are separated by a ‘|’ character. Files may be xrb archives, xsr source files, or directories of same.
–generated-sphinx-formulas-directory:  Generated XBRL formula linkbases directory.  (If absent, formula linkbases save in sphinx files directory.)

xbrlDB

–store-to-XBRL-DB:  Store into XBRL DB.  Provides connection string: host,[port], user, password, database, [timeout], technology:
postgres – XBRL-US Postgres SQL
mssqlSemantic – Semantic MSSQL SQL
mysqlSemantic – Semantic MySQL SQL
orclSemantic – Semantic Oracle SQL
pgSemantic – Semantic Postgres SQL
rexster – Rexter (Titan Cassandra)
rdfDB – RDF (Turtle, NanoSparqlServer)
json – JSON (JSON, MongoDB)
For rdfDB if hostname is rdfTurtleFile or rdfXmlFile, then database specifies a local file system pathname to store a file.  For json, if hostname is jsonFile then database specifies a local file system path name to store into a file.
Examples:
Store into postgres database using XBRL-US schema:
arelleCmdLine -f c:\temp\test.rss -v –disclosureSystem efm-pragmatic-all-years –store-to-XBRL-DB “myhost.com,8084,pgUserId,pgPassword,test_db,90,postgres”
Store into local MSSQL database using SQLEXPRESS instance:
arelleCmdLine -f c:\temp\test.rss -v –disclosureSystem efm-pragmatic-all-years –store-to-XBRL-DB “localhost\SQLEXPRESS,,sqlLogin,sqlPassword,,90,mssqlSemantic”

Disclosure System Selections

The disclosure systems are documented in the config directory file disclosuresystems.xml.  These are current entries:
  • fsa: Japan FSA
Japan FSA example entry
Default language Japanese
EDInet identifier patterns
Allowed references TBD
  • ifrs: IFRS-Example
IFRS Example
Default language English
EDInet identifier patterns
Allowed references TBD
  • sbr-nl: SBR-NL
SBR Netherlands
Default language Dutch
EDInet identifier patterns
Allowed references TBD
  • hmrc: UK HMRC (Joint Filing Validation Checks)
UK HMRC Joint Filing Common Validation Checks
Default language en-UK
Disallowed references are processed
  • efm-strict: US SEC (Edgar Filing Manual, Strict)
US SEC Edgar Filing Manual, 2013
Default language en-US (en allowed in some cases per EFM)
CIK identifier patterns
Allowed references https://www.sec.gov/info/edgar/edgartaxonomies.shtml
Disallowed references are processed
Includes content (semantic) tests
  • efm-pragmatic: US SEC (Edgar Filing Manual, Pragmatic)
US SEC Edgar Filing Manual, 2013
Default language en-US (en allowed in some cases per EFM)
CIK identifier patterns
Allowed references https://www.sec.gov/info/edgar/edgartaxonomies.shtml
Disallowed references are blocked and not loaded (same as SEC production s
ystem)
No content (semantic) tests are reported
  • efm-strict-all-years: US SEC (Edgar Filing Manual, Strict, all years)
US SEC Edgar Filing Manual, 2012
Default language en-US (en allowed in some cases per EFM)
CIK identifier patterns
Allowed references https://www.sec.gov/info/edgar/edgartaxonomies.shtml
Disallowed references are processed
Includes content (semantic) tests
  • efm-pragmatic-all-years: US SEC (Edgar Filing Manual, Pragmatic, all years)
US SEC Edgar Filing Manual, 2012
Default language en-US (en allowed in some cases per EFM)
CIK identifier patterns
Allowed references https://www.sec.gov/info/edgar/edgartaxonomies.shtml
Disallowed references are blocked and not loaded (same as SEC production system)
No content (semantic) tests are reported
  • gfm-us: US SEC (Global Filing Manual)
US SEC interpretation of Global Filing Manual
Default language en
CIK identifier patterns
Allowed references https://www.sec.gov/info/edgar/edgartaxonomies.shtml
Disallowed references are processed

Batch Script Files

A set of batch script files are provided for convenience of running repetitive things.  Contributors are encouraged to improve them and make them suitable for Mac and Linux (originally just Windows):
  • runEFMTests.bat:  Runs all of the EFM tests in the indicated source tree of test files.  Outputs log with error messages, and an Excel-compatible file (.csv for the moment) with the same appearance as the tests tree grid GUI view.
  • runUS-GFMTests.bat:  Same, but runs tests in the Global Filer Manual, instead of Edgar, mode.  Just for experimentation at this moment.
  • runXDTTests.bat:  Runs the XDT conformance suite, and outputs messages log and Excel report.
  • runGenerateVersioningTestcases.bat:  Generates a versioning creation/consumption test suite from the excel index file, such as 1000-2000-index.xml.
  • runVersioningConsumptionTests.bat:  Executes versioning consumption testcases from the index file noted in the settings.
 

68 Responses to Command Line Operation

  1. Harold Kinds says:
    August 17, 2011 at 4:26 am

    I have tested the cmdline:

    – loading the msft instance works OK
    sw: arellecmdline -f msft\msft-20110630.xml
    [info] loaded in 4,20 secs at 2011-08-17T13:19:31

    – validating the msft instance works OK
    sw: arellecmdline -f msft\msft-20110630.xml -v
    [info] loaded in 4,19 secs at 2011-08-17T13:19:47
    [info] validated in 5,22 secs

    – exporting the DTS to a csv works OK
    sw: arellecmdline -f msft\msft-20110630.xml –csvDTS dts.csv
    [info] loaded in 4,19 secs at 2011-08-17T13:20:56

    – but exporting the pre (same for Dim and Cal): not OK!
    sw: arellecmdline -f msft\msft-20110630.xml –csvPre pre.csv
    Traceback (most recent call last):
    File “c:\python32x86\lib\site-packages\cx_Freeze\initscripts\Console3.py”, line 27, in
    File “arelleCmdLine.py”, line 11, in
    File “C:\Users\Herm Fischer\Documents\mvsl\projects\Arelle\ArelleProject\src\arelle\CntlrCmdLine.py”, line 124, in main
    File “C:\Users\Herm Fischer\Documents\mvsl\projects\Arelle\ArelleProject\src\arelle\CntlrCmdLine.py”, line 233, in run
    File “C:\Users\Herm Fischer\Documents\mvsl\projects\Arelle\ArelleProject\src\arelle\ViewCsvRelationshipSet.py”, line 12, in viewRelationshipSet
    TypeError: __init__() takes exactly 5 arguments (4 given)
    [info] loaded in 4,19 secs at 2011-08-17T13:21:56
    sw:

    Reply
    • admin says:
      August 17, 2011 at 10:20 pm

      Thank you for testing this, the -csvPre command line option is fixed

      Reply
      • 8maki says:
        August 19, 2011 at 10:25 pm

        Have you already fixed this issue?
        I have the same error on the latest code.


        python3 -m arelle.CntlrCmdLine -f ../files/tdnet-qcedjpfr-24320-2011-06-30-01-2011-07-29.xbrl --csvPre ../pre.csv
        [info] loaded in 15.84 secs at 2011-08-20T14:21:37
        Traceback (most recent call last):
        File "/usr/local/Cellar/python3/3.1.3/lib/python3.1/runpy.py", line 128, in _run_module_as_main
        "__main__", fname, loader, pkg_name)
        File "/usr/local/Cellar/python3/3.1.3/lib/python3.1/runpy.py", line 34, in _run_code
        exec(code, run_globals)
        File "/Users/yamakiwataru/Workspace/taiga/xbrl/Arelle/arelle/CntlrCmdLine.py", line 248, in
        main()
        File "/Users/yamakiwataru/Workspace/taiga/xbrl/Arelle/arelle/CntlrCmdLine.py", line 115, in main
        CntlrCmdLine().run(options)
        File "/Users/yamakiwataru/Workspace/taiga/xbrl/Arelle/arelle/CntlrCmdLine.py", line 222, in run
        ViewCsvRelationshipSet.viewRelationshipSet(modelXbrl, options.csvPre, "Presentation", "http://www.xbrl.org/2003/arcrole/parent-child")
        File "arelle/ViewCsvRelationshipSet.py", line 12, in viewRelationshipSet
        view = ViewRelationshipSet(modelXbrl, csvfile, header)
        TypeError: __init__() takes exactly 5 positional arguments (4 given)

        Reply
        • admin says:
          August 20, 2011 at 11:09 am

          I see you are running from source code, I believe this fix (in gitHub) should have fixed that: “fix lxml conversion issues in CSV Relationships output”, 2011-08-17 15:55:07

          modelXbrl.modelManager.showStatus(_(“viewing relationships {0}”).format(os.path.basename(arcrole)))
          – view = ViewRelationshipSet(modelXbrl, csvfile, header)
          + view = ViewRelationshipSet(modelXbrl, csvfile, header, lang)
          view.view(arcrole, linkrole, linkqname, arcqname)

          Please send an e-mail to support@arelle.org if I can help in more detail to debug this issue! Thanks.

          Reply
      • Harold Kinds says:
        August 26, 2011 at 3:40 am

        Thanks, works ok now.
        Don’t forget to sync the documentation on this page with the -h contents!

        Reply
  2. Don says:
    October 30, 2011 at 3:50 pm

    Missing --csvFacts=CSVFACTLIST in this doc.

    Reply
  3. Ignacio Santos says:
    November 10, 2011 at 12:50 pm

    I can get in a csv file the dimensions, the primary items. Is it possible to get the compnents of the formulas, parameters, filters, inputs, and so on, in a csv file?

    Regards,

    Ignacio Santos

    Reply
    • admin says:
      November 10, 2011 at 1:07 pm

      It would be quick to add the csv equivalent of the formula GUI treeview pane, would that be helpful for your needs?

      (And then both can be improved with more details about the formula LB resources and arc parameters, as needed.)

      Reply
    • admin says:
      November 27, 2011 at 2:00 pm

      –csvFormulae parameter added (with corresponding csv output equivalent to GUI formula tree view)

      Reply
      • Ignacio Santos says:
        April 1, 2012 at 4:40 am

        Example of formulas in bach (windows 7):

        c:

        cd\

        cd \Program Files\Arelle

        echo on

        arelleCmdLine.exe -f “C:\Taxonimy\FINREP_2008\es-be-FINREP_Informes\IS1_6610.xbrl” –htmlFormulae=”C:\Taxonomy\XBRLFormula6610.CSV”

        Reply
  4. FundXJim says:
    December 1, 2011 at 5:26 am

    In the GUI, you can see a fact table with concepts and date dimensions and values. You can copy & paste this table to excel. Is it possible to output this same view to a csv file via the command line?

    Very useful app by the way

    Cheers

    Jim

    Reply
    • admin says:
      December 1, 2011 at 11:47 am

      Yes, I added descriptions of –csvFacts and –csvFactCols to the command line documentation page (they were in the source code and –help text). There’s an example in the windows build, file exportCsvFromXbrlInstance.bat, including the dimensions, which I’ve cut and pasted below (with some file paths from my laptop, you’d need to edit this if for the Mac):

      rem Export CSV from XBRL Instance with Dimensions
      rem Please edit or adapt to location of instance documents, output files, and Arelle installation
      @set XBRLINSTANCEROOT=C:\Users\Herm Fischer\Documents\mvsl\projects\EuroFiling\CSV converter\taxonomy2\taxonomy\eu
      @set INSTANCEFILE=%XBRLINSTANCEROOT%\instance.xbrl
      @set OUTPUTLOGFILE=%XBRLINSTANCEROOT%\conversion-log.txt
      @set OUTPUTCSVFILE=%XBRLINSTANCEROOT%\converted-instance.csv
      @set ARELLE=c:\Program Files\Arelle\arelleCmdLine.exe
      “%ARELLE%” –file “%INSTANCEFILE%” –csvFactCols “Label unitRef Dec Value EntityScheme EntityIdentifier Period Dimensions” –csvFacts “%OUTPUTCSVFILE%” 1> “%OUTPUTLOGFILE%” 2>&1

      Reply
  5. Naveen says:
    December 15, 2011 at 7:45 am

    I am unable to get the –csvTestReport option to work. I tried running the runEFMTests.bat test batch provided against an instance document. It only put out the logFile.

    Reply
    • admin says:
      December 28, 2011 at 9:15 pm

      The messages log result that you got is what is intended from a single instance document. The test report, on the other hand, is a result from running an XBRL testcases file or testcases index, which usually represents a suite of tests, indicating test case variations, what to load, expected results to pass, etc. If running a whole suite (such as the conformance suite for base spec, XDT, or other), then the test report shows each test, the expected and achieved result, whether the test passes or not, etc. But on a single instance, just the log of messages applies.

      Reply
  6. Mikey says:
    January 12, 2012 at 5:35 pm

    In the GUI it is possible to switch the label language and also change label format to name and to Standard Label. Is there a way to get all of those formats as separate columns when calling through command-line?

    Reply
    • admin says:
      January 12, 2012 at 5:58 pm

      yep, easy addition, will also add same to the new web interface (it’s common code), wondering, which views are you asking for first (e.g., fact list, tree views, etc)?

      Reply
  7. Mikey says:
    January 12, 2012 at 6:14 pm

    I am calling the webserver with …facts?media=xhtml&factListCols=Label,contextRef,unitRef,Dec,Value,EntityScheme,EntityIdentifier,Period,Dimensions. Right now to get the different language labels, I would have to change the system language and call it again.

    Reply
  8. Carey says:
    January 16, 2012 at 4:06 am

    Hi, thanks so much for this I am looking to implement a little validation tool, and I have two questions really,
    1. I was wondering if it was possible to validate multiple xbrl instances via the command line?
    2. Currently it takes between 20 to 30 seconds to load each instance and then a few seconds to validate it. Would it be possible to cache the xsd, linkbases, etc and use this to validate xbrl instances via the command line?

    Any help would be brilliant, thanks
    Carey

    Reply
    • admin says:
      January 16, 2012 at 8:23 am

      Yes, but in stages of difficulty. It largely depends on your type of instances.

      1) If they are like US-SEC or IFRS, with each instance having independent extensions and linkbases, there is much less to be gained and significant work to preserve in memory what might be shared.

      2) If the instances all have precisely the identical linkbases and taxonomies, then it is quite easy and a lot to gain in speed.

      3) Several others have found that by profiling the validations, we’ve been able to get the speed improved significantly, if the instances can be tested on our end for this, please contact support@arelle.org by private message.

      4) There might be a lot more flexibility in sharing using the web service API, because the command line invocations are separate of each other and don’t share any process or memory. (Or a design could consider a list of multiple instances all sharing the same DTS in one command line call.)

      Reply
  9. XbrlPassion says:
    February 4, 2012 at 2:24 pm

    error message from bat file:

    Usage: arelleCmdLine.exe [options]

    arelleCmdLine.exe: error: no such option: –csvFactCols

    Any idea?

    Reply
  10. XbrlPassion says:
    February 4, 2012 at 3:08 pm

    Some more info. I have tried both the 32 and 64 bit version of the Windows Arielle Command Line. Neither of them take the –csvFactList as an option. Any idea when this will be supported?

    Regards,

    Charles

    Reply
    • admin says:
      February 4, 2012 at 11:49 pm

      In doing the web services API, the file output was enhanced from just csv, to also html, text, xml, and json (based on the output file name extension), so the parameter was renamed from csvFactList to factListCols. The –help option should have had the latest supported by the code, but anyway the web page and bat file needed editing. Thanks for digging into this. The bat files examples and web page have been updated.

      Reply
  11. Niels-Peter rønmos says:
    May 10, 2012 at 1:30 am

    Great tool! Works with the DK-GAAP.

    I’ve tested some of the arguments. But it doesn’t work when using:
    arelleCmdLine.exe -f “P:\dcca20120101\20120101\entryDanishGAAPBalanceSheetAccountFormIncomeStatementByFunctionIncludingManagementsReviewStatisticsAndTax20120101.xsd” -i “c:\temp\npn.xbrl” -v –facts “c:\temp\npn.csv” –factListCols “Label Name contextRef unitRef Dec Prec Lang Value EntityScheme EntityIdentifier Period Dimensions”

    The Name colnne is blank! (I assume that “Name” is the name off the Element from the taxonomy.

    Reply
  12. Mauricio Ahumada says:
    July 3, 2012 at 1:10 pm

    Hi, I’m having problems running the following command line

    C:\Program Files\Arelle>arelleCmdLine.exe -f “C:\Users\mauricio.ahumada\Desktop\
    XBRL 2012-03\Estados_financieros_(XBRL)60806000_201203.xbrl” -v

    I just got several errors similar to the following:

    [] HTTP Error 404: Not Found
    retrieving http://xbrl.ifrs.org/taxonomy/2011-03-25/generic-link.xsd –
    [FileNotLoadable] File can not be loaded: http://xbrl.ifrs.org/taxonomy/2011-03-
    25/generic-link.xsd – http://xbrl.ifrs.org/taxonomy/2011-03-25/full_ifrs/ifrs_7_
    2011-03-25/gre_ifrs_7_2011-03-25.xml 12
    [] HTTP Error 404: Not Found
    retrieving http://xbrl.ifrs.org/taxonomy/2011-03-25/generic-reference.xsd –
    [FileNotLoadable] File can not be loaded: http://xbrl.ifrs.org/taxonomy/2011-03-
    25/generic-reference.xsd – http://xbrl.ifrs.org/taxonomy/2011-03-25/full_ifrs/if
    rs_7_2011-03-25/gre_ifrs_7_2011-03-25.xml 13

    When using GUI mode, loading and validating XBRL works fine, problem arises from whitin the command line mode.

    Greetings!

    Reply
    • admin says:
      September 6, 2012 at 7:37 am

      It looks like these files are not on the ifrs website at this moment. If you have them they could be copied into the cache (on Windows, using tools->internet->manage cache to locate the right directory, or handled by the mappings.xml file in the config directory. (For further help please contact support@arelle.org.)

      Reply
  13. Mike says:
    July 31, 2012 at 8:47 pm

    Hey, I seem to get a syntax error when trying to run the command line. Any ideas, or would it be something I’m doing wrong?


    mike@mike-VirtualBox:~/arelle/Arelle$ python arelleCmdLine.py
    Traceback (most recent call last):
    File "arelleCmdLine.py", line 10, in
    from arelle import CntlrCmdLine, CntlrComServer
    File "/home/mike/arelle/Arelle/arelle/CntlrCmdLine.py", line 14, in
    from arelle import (Cntlr, FileSource, ModelDocument, XmlUtil, Version,
    File "/home/mike/arelle/Arelle/arelle/Cntlr.py", line 409
    print(logEntry, file=sys.stderr)
    ^
    SyntaxError: invalid syntax

    Thanks.

    Reply
  14. Nils Wilhelm says:
    March 5, 2013 at 2:32 pm

    Hi there,

    i’ve just tested the –csvPre option and it works just fine. It prints out the standard label. What i would need is the concept name and the namespace to get the exact concept. Would that be possible?

    Thanks for your help in advance

    Reply
    • admin says:
      March 6, 2013 at 3:08 pm

      for –csvPre, there’s an option to specify custom label roles, and one of those options provides prefixed name instead of label (but not the full namespace). Please try –labelRole XBRL-concept-name. Otherwise we could code a feature to have customizable columns (like the fact list –factListCols) and include a full namespace and name as well.

      Reply
  15. Mauricio Ahumada says:
    May 8, 2013 at 10:05 am

    Hi. First of all, great work, your tool is excellent.

    Im having some trouble getting a validation report when calling Arelle through command line. I dont seem able to find a way to write a report showing the errors the XBRL instance has.

    I execute this

    C:\Program Files\Arelle>arelleCmdLine.exe -f c:\DBNeT\xbrl\out\XBRLMALOS\993010002_201303_I.xbrl -v

    The program finishes and exit with to message, output or anything. I know for certaint the instance has some errors, in fact the graphical version of Arelle detects correctly such inconsistencies.

    Any help?

    Thanks in advance.

    Reply
  16. Jetnor Arifaj says:
    March 6, 2014 at 7:10 am

    Hi, Thanks for the very good overview of command line. I am wandering if there is a way to save a view from individual sheets to HTML using command line. The
    reason I ask is because some of the instances are too big to load in the gui version.
    Using the GUI version of Arelle it gives you the option on the
    save as to change the file type to HTML table which is quite useful for viewing. can this be done using the command line? Can you please share an example?
    Many thanks, Jetnor.

    Reply
    • admin says:
      March 10, 2014 at 10:47 pm

      This depends on what you mean by “sheets”. The link bases can be saved individually. Table link base can be saved to html files (there is an EBA plugin which saves each to a separate file in a batch operation). Please clarify, suggest using support@arelle.org for further details.

      Reply
  17. Marinos says:
    May 10, 2014 at 2:48 pm

    Sorry for the basicness of the question but i am not experienced with coding and i really need to use arelle…

    I give this simple command in cmd and arelle parses the data i want and stores them in example.html

    python arelleCmdLine.py -f http://www.sec.gov/Archives/edgar/data/1009672/000119312514163161/crr-20140331.xml -v –factTable example.html

    The only problem is that i dont know where arelle saves example.html. Sometimes it saves the file in the C:\a which is the arelle directory on my PC and other times it saves it on the Documents library.

    How can i specify a certain directory where the html files will be stored by modifying this command in the cmd?

    Arelle is Suberb!!!!!
    Thank you for sharing such an amazing tool!

    Reply
  18. Jason says:
    May 12, 2014 at 8:41 am

    I am trying to use the command line to automate pulling new files from the Edgar RSS feeds. I can get the zip file from the feed, but I am having trouble figuring out the right arguments to process a zip file through the command line.

    What would a sample command line operation look like using zip files? Or is that even possible?

    Reply
  19. Marinos says:
    May 25, 2014 at 10:04 am

    Dear admin

    I am a finance enthusiast and i will like to ask if there is any way to use arelle to import values you choose for example: Import Cash and cash equivalents and its numeric value to a cell that is designated. I am using –factTable to pull all the facts but scanning with Excel so many different document types of companies proves very hard. Is there anyway to immediately import values with arelle to designated cells in arelle.

    Thank you

    Reply
  20. Marinos says:
    May 28, 2014 at 8:34 am

    Dear Herm Fischer

    Can i use the amazing function Find [Text (ignore case)] when i am running arelle from source? And save those data inside a csv or html or xls file?

    I couldn’t find it on the documentation.

    Best regards
    Marinos

    Reply
  21. Fin_Edw says:
    June 16, 2014 at 2:02 pm

    Dear admin

    We are college students in Stockholm. How is it possible to extract one specific piece of data e.g.: Revenues when the corresponding tag in XBRL can vary; it can be: us-gaap:Revenue, us-gaap:SalesRevenueNet, etc. Is there a way to know the number of names that can be used in XBRL for Revenue, Net Income etc from somewhere (before the query) in order to address issues of data transparency?

    Reply
  22. David Jarvis says:
    June 22, 2014 at 5:04 pm

    Hello there!

    This is quite an impressive endeavor – I’ve been digging into it a little bit and you’ve done a ton of hard work here.

    I’m trying to convert an instance document to json, which I believe I should be able to do a command like the following:

    python3 arelleCmdLine.py -f ../xbrl/data/0001193125-14-058842-xbrl/aol-20131231.xml -v –store-to-XBRL-DB “jsonFile,port,user,password,/Users/davidjarvis/demo,timeout,json”

    However, I get an extremely lengthy stacktrace back:
    [] [Exception] Failed to complete request:
    Type gYear is not supported for json output
    [‘ File “/Users/davidjarvis/Development/Python/Arelle/arelle/CntlrCmdLine.py”, line 830, in run\n pluginXbrlMethod(self, options, modelXbrl)\n’, ‘ File “/Applications/Arelle.app/Contents/MacOS/plugin/xbrlDB/__init__.py”, line 205, in xbrlDBCommandLineXbrlRun\n storeIntoDB(dbConnection, modelXbrl)\n’, ‘ File “/Applications/Arelle.app/Contents/MacOS/plugin/xbrlDB/__init__.py”, line 166, in storeIntoDB\n result = insertIntoDB(modelXbrl, host=host, port=port, user=user, password=password, database=db, timeout=timeout, product=product, rssItem=rssItem, **kwargs)\n’, ‘ File “/Applications/Arelle.app/Contents/MacOS/plugin/xbrlDB/XbrlSemanticJsonDB.py”, line 60, in insertIntoDB\n jsondb.insertXbrl(rssItem=rssItem)\n’, ‘ File “/Applications/Arelle.app/Contents/MacOS/plugin/xbrlDB/XbrlSemanticJsonDB.py”, line 276, in insertXbrl\n self.commit(g)\n’, ‘ File “/Applications/Arelle.app/Contents/MacOS/plugin/xbrlDB/XbrlSemanticJsonDB.py”, line 230, in commit\n self.execute(“Saving RDF Graph”, graph=graph)\n’, ‘ File “/Applications/Arelle.app/Contents/MacOS/plugin/xbrlDB/XbrlSemanticJsonDB.py”, line 183, in execute\n default=jsonDefaultEncoder)) # might not be unicode in 2.7\n’, ‘ File “/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/__init__.py”, line 240, in dumps\n **kw).encode(obj)\n’, ‘ File “/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py”, line 193, in encode\n chunks = list(chunks)\n’, ‘ File “/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py”, line 414, in _iterencode\n for chunk in _iterencode_dict(o, _current_indent_level):\n’, ‘ File “/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py”, line 388, in _iterencode_dict\n for chunk in chunks:\n’, ‘ File “/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py”, line 388, in _iterencode_dict\n for chunk in chunks:\n’, ‘ File “/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py”, line 388, in _iterencode_dict\n for chunk in chunks:\n’, ‘ File “/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py”, line 388, in _iterencode_dict\n for chunk in chunks:\n’, ‘ File “/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py”, line 388, in _iterencode_dict\n for chunk in chunks:\n’, ‘ File “/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py”, line 388, in _iterencode_dict\n for chunk in chunks:\n’, ‘ File “/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py”, line 388, in _iterencode_dict\n for chunk in chunks:\n’, ‘ File “/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py”, line 422, in _iterencode\n o = _default(o)\n’, ‘ File “/Applications/Arelle.app/Contents/MacOS/plugin/xbrlDB/XbrlSemanticJsonDB.py”, line 133, in jsonDefaultEncoder\n raise TypeError(“Type {} is not supported for json output”.format(type(obj).__name__))\n’] –

    I’m not sure what it means by “type gYear is not supported for json output” — any insights or potential fixes?

    Thanks a million 🙂

    – David

    Reply
  23. Robert says:
    August 14, 2014 at 12:16 am

    Hello,
    first of all I would I have to say, that Arelle is a great tool. Being able to display XBRL instances as proper tables and converting them to csv etc. is great. But currently I am struggling to achieve a certain output as csv.
    In the table view in the GUI I can see the table name and the row and column number of a certain value. If I use the following command I can extract the dimensions etc.

    C:\Program Files\Arelle>arelleCmdLine.exe -f CO_LCR_Ind.xbrl –facts=D:\flc.csv –factListCols=”Label Name contextRef unitRef Dec Prec Lang Value EntityScheme EntityIdentifier Period Dimensions”

    Instead of extracting the dimensions I would like to extract all the information (labels, row number, column number etc.) I can see in the table view in the GUI. Is that somehow possible?

    Best regards,
    Robert

    Reply
  24. anonym says:
    December 15, 2014 at 2:39 am

    Hi! I want to validate an xbrl instance file which seems to work alright. But i cant find any message containing validation results in detail. I only get overall log entries like this:

    [] Formula xpath2 grammar initialized in 2,06 secs –
    [info:profileActivity] … custom function checks and compilation 1.23 secs –
    [info:profileActivity] … assertion and formula checks and compilation 72.39 se
    cs –

    I’ve tried the REST API and this way I was able to produce more detailed output but it wasnt possoble to see which fact is causing the message. Example:

    2014-12-09 14:49:34,965 [formula:trace] Value Assertion S.27.01.01_DD3
    S.27.01.01_DD3
    Expression: if (iaf:numeric-equal(($v_0), 0)) then (true()) else (iaf:numeric-equal(($v_1), iaf:numeric-divide($v_2,$v_3)))
    Evaluated: if (iaf:numeric-equal((-2.896.740.693.461,66), 0)) then (true()) else (iaf:numeric-equal((-2.899.403.573.185,80), iaf:numeric-divide(-2.898.072.133.323,73,-2.896.740.693.461,66)))
    Result: False – http://eiopa.europa.eu/eu/xbrl/s2md/fws/solvency/solvency2/2014-07-23/val/val-s.27.01.01.xml 5792

    So my basic question would be how to access validation results of an xbrl instance so that it’s possible to identify which facts where causing a validation error?

    Regards,
    A

    Reply
    • admin says:
      February 21, 2015 at 5:24 pm

      Suggestion is to try with xml output, the xml (and JSON) results have much more details. I’ve only tested this with python-coded validations, so if the formula output isn’t identifying the fact we can add further result details to the xml results.

      Reply
  25. Nils Wilhelm says:
    January 27, 2015 at 6:02 am

    Hi there,

    I’m working with the command line tool and figured out how to export the concepts, presentation and calculation linkbase. I was wondering if there is no option to export labels and references. As far as I know the webserver interface has this feature. Any help would be highly appreciated!

    Best regards,

    Nils

    Reply
  26. Ankush says:
    April 28, 2015 at 5:33 am

    I am trying to run HMRC validations on a iXBRL document through web service. Kindly let me know if the following parameters are correct, as I am not able to get the desired output.

    localhost:8080/rest/xbrl/myFilePath/validation/xbrl?media=json&calcDecimals&utr&hmrc&

    where myFilePath is my xhtml file path.

    Reply
  27. Rathesh says:
    May 29, 2016 at 9:55 pm

    Hi,

    In arelle GUI working and Show the validation rules dqc_us_0005 as well as dqc_us_0006 rules.
    But In command Line running as a web server its showing the validation rules of dqc_us_0006 only, not showing dqu_us_0005.

    Is there any possible way to fix this issue.Thank in advance.

    Reply
  28. Rathesh says:
    May 29, 2016 at 10:08 pm

    Hi,
    Is there any difference between validate of GUI and validation in Command Line Interface? why because validation of DQC running in both GUI and Command Line But GUI gives me result what i excepted but Command Line not.

    please provide suggestions…

    Thank you…

    Reply
  29. Rathesh says:
    June 2, 2016 at 9:11 am

    Hi,

    How to validate DQC Rules in Cmd Line …

    Thank you …

    Reply
  30. Prk says:
    August 28, 2016 at 11:26 am

    Is there any method so that the command line output in csv could be flattened, thanks in advance

    Reply
  31. Rafal says:
    September 9, 2016 at 1:52 am

    Hello,

    Have some problems with ArelleCmdLine on Windows machine. I developed my project on MacOS and was using Arelle cmd line app + modified saveLoadableExcel plugin. Everything worked ok. But now I’m asked to run it on Windows machine and can’t make it work. When I execute:

    C:\ProgramFiles\Arelle\ArelleCmdLine.exe -f report.xml -v –save-loadable-excel

    i always get:

    Usage: Program [options]

    Program: error: no such option: –save-loadable-excel

    Tried to add plugin via GUI and through cmd but still get the same output.

    Is that known issue or am I doing something wrong?

    Reply
  32. Abhijit says:
    September 15, 2016 at 1:49 am

    I am struggling to create a DOS command line which takes input XBRL file and converts to CSV. I need entire data in an output csv file with logging. Help would be appreciated.

    Reply
  33. William says:
    July 17, 2017 at 6:20 am

    It is obvious that Arelle’s interface is not user friendly. It seems the open source project was artificially made difficult to be used, except paying money, or you can’t get any help.

    Reply
    • admin says:
      July 18, 2017 at 11:07 am

      This is an open source project, original developers don’t ask for money and are usually very busy. If you have suggestions on friendliness, please send to support@arelle.org or arelle-users@googlegroups.com. There are also commercial support organizations that have support bandwidth in specific markets, and of course they do need to pay rent and salary.

      Reply
  34. Rinsu Skariah says:
    January 13, 2020 at 2:44 am

    Hi, I am using 19.2 Arelle , I am getting below error when validating using command line
    Exception loading plug-in Validate EFM: No module named ‘holidays’

    CommandLine ::

    arelleCmdLine.py -f “D:\PythonXbrl\re3\Renderer192\Test\in” –disclosureSystem “efm-pragmatic” –plugins “D:\PythonXbrl\re3\Renderer192\arelle\plugin\validate\EFM|validate/EFM” –logFile “D:\PythonXbrl\re3EdgarRendering\Renderer192\Test\out\ArelleInlineXbrlValidationResult.xml”

    Reply
  35. Rinsu Skariah says:
    January 27, 2020 at 4:12 am

    Resolved holiday library has to be installed. Thanks

    Reply
  36. Shubham Asolkar says:
    March 14, 2021 at 11:05 pm

    How can I change location of web cache when running through command line ? is it done through –xdgConfigHome option ?

    Reply
    • admin says:
      March 18, 2021 at 4:04 pm

      –xdgConfigHome: Specify non-standard location for configuration and cache files (overrides environment parameter XDG_CONFIG_HOME, if also provided). (Can also do with XDG_CONFIG_HOME environment parameter.)

      Reply

Leave a Reply to admin Cancel reply

Your email address will not be published. Required fields are marked *

en English
ar Arabiczh-CN Chinese (Simplified)nl Dutchen Englishfr Frenchde Germaniw Hebrewit Italianko Koreanpl Polishpt Portugueseru Russianes Spanish