Author: kris

Adobe Automation – work in progress

A few months ago, I started work on new infrastructure to automate Adobe Creative apps (and other apps).

A quick update: work is still steadily progressing.

At the the moment, I have built a framework in pure C++ 11, called ‘Tightener’.

There are no external dependencies – all you need to compile it is a C++ 11 compiler and a standard C++ library.

Tightener has an embedded language, called TQL (Tightener Query Language), pronounced as ‘tickle’.

TQL is similar to JavaScript, but simplified. It has the basics: expressions, functions, scopes, cooperative multitasking….

Tightener provides support for ‘Tightener Nodes’ which are instances of Tightener that are embedded into some environment (a command-line console, a language interpreter, an application, a network stack…).

Tightener nodes running on the same computer can communicate with one another using pipes/shared memory.

I envision many nodes per computer. There is one ‘main’ node and number of satellite nodes embedded into various environments, providing services (e.g. Python, InDesign, Illustrator, node.js…)

Using the messaging mechanism, one Tightener node can ask another Tightener node to run a TQL script.

TQL provides support for some XPath-like queries into entity-attribute Object Models – think of it as ‘XPath for JSON’.

Most embedded Tightener nodes will be able to handle queries into some object model (e.g. a Tightener node embedded in InDesign will handle queries into the InDesign DOM), and hence allow other Tightener nodes to make queries on their behalf.

I am currently integrating Tightener into an InDesign plug-in, so the InDesign DOM will become an integral part of the InDesign Tightener node.

Once that is done, any other Tightener node on the same computer can send TQL scripts to the InDesign node to query or execute.

Next I’ll integrate Tightener into Python, so I can start showing proof-of-concept demos: open a Jupyter Notebook, and interactively construct an InDesign automation script.

TQL is not meant for full-fledged programming (though it is powerful enough to do that).

TQL’s main goal is to reduce the number of round trips between the controlling Tightener node and controlled Tightener node: there is latency between the controller and the controlled. If we can execute some coded logic on the controlled node, that helps reduce the number of round trips we need.

(In short: this is the same thing as why ExtendScript is much faster than AppleScript. AppleScript has some latency each time we ‘cross the fence’ between the controlling AppleScript and the controlled program. ExtendScript lives ‘inside the fence’ and grafts directly only the controlled program).

A remote Tightener node is more ‘like AppleScript’.

An embedded Tightener node is more ‘like ExtendScript’.

So we want any remote Tightener nodes to delegate some of their work to an embedded Tightener node whenever possible. Hence TQL.

TQL is currently fleshed out enough to run all kinds of scripts, but there is still a lot of unimplemented things on my ‘to do’ list – e.g. the ‘switch’ statement.

Tightener nodes are, by design, not network-aware.

Networking functionality has to be achieved by building separate ‘gateway nodes’ which are essentially one or the other network comms stack with a copy of Tightener embedded.

This approach is to avoid introducing unwanted dependencies into Tightener: I don’t want to have to upgrade Tightener each time a new version of OpenSSL is released.

Next steps:
– Finalize Tightener integration into InDesign
– Embed Tightener into Python
– Create a Tightener gateway node by combining Tightener with OpenSSL

I’ve started a public Github repo where I’ll put the Tightener documentation as I write things up.

https://github.com/zwettemaan/TightenerDocs/wiki

Snippets

There is too much to remember, and I find myself searching for the same information over and over.

I’ve got a bunch of loose tidbits in Evernote, but in practice, Evernote is a bit of a sink: file a lot, rarely retrieve.

I am going to start keep track up useful snippets of code.

Every time I do a search and find some samples, I’ll make a re-usable snippet out of it – sample code for copy-pasting or testing

I’ll try to keep them organized, mainly for my own use, and for anyone else who can use it.

I’ll create a separate page for these: https://coppieters.nz/?page_id=451

Adobe Creative Cloud Automation

Project Status

26-Aug-2021. It’s still very early days: I am currently building the Tightener core module in C++, trying to use TDD (Test Driven Development).

I’ve got the basic message passing working, and am about to start on the IPC (InterProcess Communication) to allow nodes to discover and communicate with one another.

Once that is done, I’ll start work on a Node.js language support wrapper node, and an InDesign DOM node.

Only at that point will I be able to deliver a proof-of-concept. We’re nowhere near that point yet. Stay tuned.

UI often comes last

When it comes to automating the Adobe Creative Cloud apps, the first concern I have is: how to interact with the document.

What does the DOM (Document Object Model) look like?

For the work I do, the UI is but an optional wrapper around the core functionality, and many of the tools I develop don’t interact with a user – they’re faceless components running as part of a workflow. They’re often throw-away software.

So, more often than not, the last thing I’ll think about is ‘how to create a UI?’ (User Interface) for the automation components I am building.

Even if a UI is warranted, what good is a UI if there is no functionality to be wrapped?

Many times, the solutions I create will have the most limited UI possible (command line, run from a debugger,…).

There is no point in building a UI for something that will only run once.

UXP: thin on the DOM side, heavy on the UI side

Adobe’s UXP (User Extensibility Platform) is heavy on the user interface end and light or non-existent on the DOM end.

For the kind of work I do, UXP is currently irrelevant.

ExtendScript: on life support

ExtendScript offers us a usable way to access the document DOMs, but ExtendScript is dying.

The available tools are suffering from neglect and one by one, they become unusable.

And then there are the breaking changes.

All too often, I have to urgently fix problems caused by breaking changes by Adobe.

Between a rock and a hard place

I think we will soon be in a situation where the old does not work any more, and the new does not work yet.

That’s called ‘being between a rock and a hard place’.

I’ve been thinking about this, and I think there is a way out for third party developers.

The goal is to provide us with what I think we need (DOM access), and reduce our exposure to breaking changes by Adobe.

This project should help limit the damage caused by breaking changes and give us a little bit more control over timely fixes.

Separation of concerns

There are a number of concerns that I keep in mind.

Programming Language

We should separate the DOM from the language used. ExtendScript forces one to use a JavaScript variant to access an app’s DOM.

If you want to use, say, Python to automate an Adobe Creative Cloud app, you’re out of luck.

That does not need to be so.

DOM access and the programming language used are two separate concerns and they should also be separated into independent parts of the automation support infrastructure.

External Dependencies

Another issue I often face is with C++ plug-ins and external dependencies. Many apps offer a plug-in architecture.

A recurring issue with those are external dependencies.

InDesign, for example, is written in C++, and has dependencies on a host of external code bodies: Boost, OpenSSL, zlib, icu…

Each time Adobe updates InDesign, my plug-ins need to be recompiled using the correct InDesign SDK for the new version of InDesign.

But when InDesign changes its own internal dependency from, say, Boost 1.65 to Boost 1.72, any plug-ins that rely on Boost need to match that dependency change.

Updating a plugin involves not just compiling against an updated InDesign SDK but also updating these additional dependencies to carefully match any depencency changes in InDesign, and that is a major time-sink.

To make updating plug-ins easier it I now think it’s better to opt for multiple smaller plugins rather than one large plug-in.

When an external dependency changes, this would only affect the plug-ins that actually need it, and the other plug-ins could be updated with much less effort.

Another approach that often works for me is to try hard to not have any external dependencies, and re-implement the needed functionality.

Rather than pulling in a massive external framework and then only use 1% of what it can do, I’d rather opt for re-implementing the functionality, and go dependency-free.

Connectivity

Another recurring concern is connectivity. We often need to automate ‘across’ a network wire.

My Current Thinking

Below is my current thinking for the architecture of the project.

The Tightener API

Tightener is the low-level ‘glue’ between the many possible components.

It is like Adobe Bridge (the protocol, not the application), but open source.

Tightener borrows ideas from Adobe Bridge and AppleEvents.

It allows message passing between the various components.

Each component that has the Tightener code embedded is a ‘node’.

Tightener is written in plain C++ 11, and has no external dependencies. It can be compiled into standalone apps and into plug-ins.

On a single workstation, we would have a number of active Tightener nodes.

There would be a single central Tightener node – the switchboard which ties all other nodes together.

Other nodes on the workstation communicate with the central node by way of IPC (InterProcess Communication)/named pipes.

Tightener would have a method of discovery and registration of satellite nodes with the switchboard node.

There would be a satellite node for each of the supported apps, by way of plug-ins with Tightener compiled-in.

There would be an InDesign node, an Illustrator node, a QuarkXPress node,… and just like Adobe Bridge, we would be able to pass messages between those nodes by way of the switchboard node.

These app nodes, which are mostly embedded in plug-ins, would then interact with the app to provide a DOM to the outside world.

If there is a need for external communication, we would have a Tightener satellite node that implements external communication.

Tightener itself would not need to ‘know’ how to communicate over a network.

For language support, there would be a Tightener node for each language we want to support – e.g. a Tightener node in Python, a Tightener node in Node.js,…

Scripts run in whatever language we desire, and use ‘their’ Tightener node to communicate with the app nodes.

It would open the door for scripters to use whatever environment they please – JavaScript, TypeScript, Python, PHP, Xojo, Swift, C++, some future language…

InDesign DOM

Separate from that, I would start a second open source project. It would be a Tightener-based InDesign DOM project.

I suspect I’ll be able to graft this onto the infrastructure used for ExtendScript – the InDesign DOM is not tied to just ExtendScript, it is also used for IDML, AppleScript and VBScript.

I suspect it might be feasible to write another layer for Tightener, and have Tightener living in the same realm as ExtendScript, IDML and AppleScript.

It would offer an alternate to the ExtendScript DOM, and it would also be a re-implementation of the features of APID ToolAssistant (aka Active Page Items), an InDesign plug-in that I wrote long time ago, in the InDesign CS days.

Node.js Tightener Wrapper

For language support, I’d start with a Node.js Tightener node.

This would also form the basis of a node for external communications – we can implement such node using Node.js (or whatever language we choose).

Illustrator and others apps

Once that works, I’d look into getting the same for Illustrator, taking inspiration from the features I had in my AIntrospector plug-in.

Other apps can follow – as the project will be open source, other people could pick up building Tightener-based plug-ins for other apps (e.g. QuarkXPress, Adobe Premiere, Scribus, ImageMagick,…).

Active Page Items

Active Page Items is old and crusty and weird, but even after all these years it offers automation features that InDesign ought to provide, but does not.

For example: creating adornments on page items, or events that fire when individual page items are modified, compiling scripts into encrypted gobbledygook, and handle licensing and demo versions of compiled scripts to help monetize scripts.

Active Page Items has a number of problems that make maintaining it cumbersome.

Each time Adobe releases a new InDesign version, getting everything to compile and link is not fun.

One problem is that it is a monolithic plug-in, like a Swiss Army knife. If one thing breaks, everything is broken.

Another problem is that it relies on third party libraries like OpenSSL, urdl, boost, zlib.

As various dependencies change over time, getting it all to work together is not straightforward.

Tightener API implemented in C++ first

The first implementation of the Tightener API would be written in C++ so it can be compiled ‘into’ various plug-ins as well as in stand-alone applications.

If for some reason C++ cannot be linked into a target environment, the Tightener API can be re-implemented in any other sufficiently powerful computer language.

DOM plug-ins

The Tightener C++ API can be compiled into various plug-ins.

Each of these plug-ins would provide access to a small part of the host app’s DOM.

Rather than have a single plug-in do ‘the whole DOM’, I feel it would be better to have many plug-ins, each doing ‘a bit of the DOM’.

That way, the task of creating these plug-ins can be divvied out and shared amongst multiple contributors.

Some plug-ins could tie into the existing scripting architecture (e.g. InDesign’s) so it should be easy (relatively speaking) to mirror what is available in InDesign ExtendScript.

Core ideas

  • Provide a dependable way for scripters to access one or more DOMs inside of the Creative Cloud (and other) apps.
  • Reduce our exposure to breaking changes made by Adobe.
  • Open Source: this needs to be done fairly quickly, and trying to do it as a one-man band will most probably fail. By making it open source I might be able to rile up enough manpower to get this done before Adobe kills us all.
  • Separation of concerns: avoid working on stuff that can be done another way, outside of Tightener. Essentially, all Tightener would do is provide an IPC socket into an app-to-be-automated.
  • Tightener would provide just ONE low-level API, i.e. just an IPC socket. No other protocols.
  • Additional protocols and languages are considered out of scope, but can easily be provided by creating wrappers around Tightener. These wrappers talk to Tightener via IPC, and provide additional ways to interact with the app-to-be-automated
  • – WebSocket wrapper
    – HTTPS wrapper
    – Python wrapper
    – JavaScript/NodeJS wrapper
    – Xojo wrapper
    …
  • The Tightener API defines an asynchronous message protocol, somewhat similar to how AppleEvents work on Mac OS X.
  • Multiple Tightener APIs need to interact with one another – e.g. there could be a single ‘main’ Tightener API running on a computer as a standalone application. This core monitor would then interact with a collection of Tightener APIs running in multiple plug-ins in the host apps being automated.
    The scripter’s scripts would interact with the ‘main’ Tightener API which would then act as a go-between to the in-plugin Tightener APIs.
  • The Tightener API C++ implementation should be reliant as little as possible on third party libraries. This to avoid version conflicts. E.g. when Adobe decides to change the Boost version number or the ICU version number used in InDesign, that should not affect a plug-in which uses the Tightener code.

Asynchronous Message Passing

Through the Tightener API, you can send messages (‘Commands’) with a data payload to some targets.

Targets can be anything in the collected DOMs: an app, a document, a box on a page, a character style, some construct specific to the app being automated, some feature in some plug-in,…

The message will be processed, and some time later, a reply will be returned.

Targets can be defined by using descriptors.

Descriptors can be either specific or non-specific.

Specific: ‘page item with ID 123 in document ‘/Users/kris/Documents/bla.indd’

Non-specific: queries into one or more apps and/or document – e.g. ‘all frames with a blue background in all of the currently open documents’. They could even span multiple apps, e.g. ‘quit all open Adobe apps’.


GetURL revisited

This page is outdated. More recent info here:

https://www.rorohiko.com/jsxgeturl

Note: If you are using version 0.0.7 or below, it will have stopped working by now because it was time bombed on December 31 of the year. Please update to version 1.x.x (see link below).

I’ve been dabbling around with ExtendScript DLLs.

These little critters run within ExtendScript and allow me to enhance ExtendScript, for all apps that support it, including InDesign Server, on Mac and Windows.

I can add whatever functionality I decide to wrap into such a DLL.

Now, I’ve just finished wrapping the C++ libcurl into such a wrapper. A full version can be downloaded here.

https://www.rorohiko.com/downloads/rr612412/JSXGetURL.0.0.9.zip

It runs till June 30 of this year.

There is nothing to install, no admin privileges needed. You just need to add a folder with a few .jsx files to your ExtendScript project, and you can simply write ExtendScript like:

#include "JSXGetURL/JSXGetURLLoader.jsx"

var getURL = JSXGetURL();

var s = getURL.get("https://www.rorohiko.com");

alert(s.substr(0,1000));

and this works in InDesign Server, ExtendScript Toolkit, Bridge, InDesign, InCopy, Illustrator or whatever else supports ExtendScript, and it works both on Mac and Windows. And it’s very fast because it’s all compiled C/C++ code.

As it is, this thing now kind of supersedes my old ventures (https://coppieters.nz/?p=133 which uses CEP/node as a ‘servant’ to ExtendScript, and does not work on InDesign Server, and https://rorohiko.blogspot.com/2013/01/geturlsjsx.html which was pure JSX, a subset of http only, no https).

Anyway: if you’re interested in this ExtendScript enhancement and have ideas or questions, contact me at [email protected]

Creative Developers Summit 2020

Two-day virtual developer summit (June 4–5)

The Creative Developers Summit has been the go-to conference for the past half-dozen years for developers creating software for Adobe Creative Cloud applications. This year, sessions will all be online for obvious reasons.

Registration includes all sessions on both days, and it’s only $95 this year, a huge discount from previous years. Because everything is online, we’re going to have speakers prerecord their sessions so you can watch them at your leisure, and then have a fixed online time for each sessions’s Q&A, where the speakers will be there to answer your questions via Zoom.

Sign up here:

https://cpn.co/g/cds2020

We’ll have some great speakers including Davide Barranca, Mike Zahorik, James Lockman, Kris Coppieters and others. Full info will be mailed to you in a few days.

If you would like to be a speaker at summit, and have some interesting topic you’d like to speak about, here’s your chance: send us a brief outline of a presentation you’d like to add to the summit. Send your proposal to [email protected]!

Note that this Summit is complementary to, not competing with, the upcoming “Adobe Creative Cloud Digital Partner Days 2020” which are happening on June 23 and 24:

https://creativecloudpartnerday2020.creativecloud.adobeevents.com

The Creative Developers Summit is (mostly) about developers talking to developers, and won’t overlap in content with Adobe’s June event.

Hope to “see” you soon!

appMap.json

Mapping out Adobe Creative Cloud apps. Core file paths, app specifiers…

Things like: indesign-15.064 but also photoshop-140.064 and premierepro-14.0. App specifiers as might be needed when debugging with VSCode are not very regular.

Note: appMap.json looks like JSON but it ain’t JSON. It’s JSON++ aka JSON-with-comments, which is not proper JSON. The Sparker app used in CEPSparker and JSXSparker can handle JSON++

The appMap.json file is somewhat readable and contains information that could be valuable for other scripters. Note: when clicking the following link, you might see error messages about invalid JSON. That’s a side-effect of JSON++.

https://raw.githubusercontent.com/zwettemaan/JSXSparker/master/Templates/appMap.json#

JSXSparker is shaping up – it now handles download-to-running-hello-world-in-2-minutes for Bridge, InCopy, InDesign, Illustrator, Photoshop, Dreamweaver, Premiere Pro.

To support Premiere Pro, JSXSparker now comes with a rudimentary script runner panel, in order to extend the Premiere Pro UI with a means to run scripts.

appMap.json is part of the included starter template configurations.

appMap.json is the ‘roadmap’ to where various Adobe apps keep their stuff.

Amongst other things, it documents what I found out about app specifiers, which are quite ‘irregular’.

appMap.json feeds JSXSparker (and CEPSparker) but it also serves as my ongoing documentation/roadmap into the innards of the Adobe Creative cloud apps

Archive for Adobe Devs slack group

https://adobedevs.com

A sore point of our Adobe Devs slack community is that this community is just a loose swarm of cooperative developers, and we have to make do with a free Slack group.

That works well, up to a point: we only get access to the most recent 10,000 messages.

Any older messages roll beyond the event horizon and disappear into a black hole.

To remove that limitation, Slack wants some money.

Fair enough, but the issue is that Slack does not cater for us. Our community does not have any ‘substance’. We’re not a company, we’re not a non-profit. We’re just a swarm of friendly, but solitary worker bees.

Slack has no facilities for individual, per-user payments. They assume every entity they deal with is some form of commercial or legal entity. Something with a postal address or incorporation address or a head office.

Slack does offer special dispensation for non-profits, but their definition of non-profit is very narrow. Our swarm ain’t it.

We’ve been humming and hawing about this for a long time, and I’ve now decided to cut through and fix the problem.

I clicked the ‘Upgrade’ button in Slack, and they reckon we have 60 active users, which would cost US$4,800 per year or $480 per month to remove the 10,000 message limitation.

My plan (now completed):
– Take out a one-month subscription for all of us. US$480 ain’t cheap and makes a dent in my finances, especially now that Covid-19 has made a lot of things grind to a halt, but I think I can wiggle it.
Until someone bit the bullet, nothing would have happened.
– Once the subscription runs, extract all messages in all channels and put them up for public access:
https://adobedevs.com/
You need a name and password which is published in the AdobeDevs Slack, in channel #archive-howtoaccess.
– Cancel the subscription again before the month is up
– All I need to do further is to regularly re-fetch a batch of the latest messages before we accumulate 10,000 additional messages. Every 2-3 months I’ll update my archive, and we should be good without any additional costs, except for my time.

So, for the month of April 2020, everyone on the AdobeDevs slack group can enjoy access to all messages.

And after that, you can access the archive here:
https://adobedevs.com/

After I did all that, Chris Ryland put out a call to ask subscribers to chip in to help me cover the US$480 subscription fee. And as it so happens: a bunch of people jumped in, and instead of being US$480 out of pocket, I ended up US$41.05 richer than when I started.

Thank you, Sergey Kritskiy, CtrlSoftware, Иван Степанов, Roland Dreger, Joost Huizinga, Brett Kizner, Derrick Barth, Chris Ryland, Creative Scripts, Joseph Portell!

InDesign 2020 CEP Panel Size Bug

Add your vote to my bug report here:
https://indesign.uservoice.com/forums/601180-adobe-indesign-bugs/suggestions/39698728-indesign-2020-15-0-1-cep-panel-size-bug-on-windows

A customer of mine reported that a CEP panel I made for them was randomly changing size and growing.

That sounded like the old bug we experienced way back, which I assumed had been fixed.

(See https://community.adobe.com/t5/get-started/cep-panel-wrong-size-in-cc2018/td-p/9660228?page=1)

I had not seen this problem in InDesign 2020, but I can now confirm: it’s still there, at least on my Windows laptop, using InDesign 2020 15.0.1, Windows 10, Microsoft Surface 4 with HiDPI.

At first I had a bit of trouble duplicating the problematic behavior; it only happens in fairly specific circumstances.
The conditions are:
– InDesign 2020 15.0.1
– Windows 10 x64
– HiDPI display
– Windows Display Scaling > 100%
I do know the HiDPI and scaling > 100% conditions are necessary, but I don’t know for sure if these conditions are sufficient, i.e. there might be additional circumstances that I’ve not yet identified.

What I’ve been able to determine is this:

  • it is related to the display scaling on Windows. I have a HiDPI display, and when the Windows Settings for Display Scaling is set > 100%, my panel goes wonky when displayed. Often, it grows every time it is opened.
  • the issue cannot be worked around with a call to CSInterface.prototype.resizeContent
  • Once wonky, stays wonky. After the panel goes haywire, setting the scaling back to 100% does NOT fix it.
  • to reset things to normal, I need to
    a) quit InDesign
    b) set the display scaling to 100%
    c) Delete the InDesign SavedData file from below %LOCALAPPDATA%\Adobe\InDesign\Version nn.0\[LANGCODE]\Caches (e.g. C:\Users\kris\AppData\Local\Adobe\InDesign\Version 16.0\en_US\Caches)
    After doing that it comes back to normal
  • The best workaround I have for now is to not use scaling. Instead, I set the display to a lower resolution (so everything stays about the same size, but becomes fuzzy), and then the panel behaves properly.
  • I don’t know if having scaling set to > 100% is sufficient as a condition to trigger the problem. I only have one machine set up to test with.

I’ve not seen this same issue on my (Retina) Macs. I suspect it’s a Windows-only issue.

My customer sent me a video, so I knew for sure the issue was real, but I had some trouble making it happen on my machines.

I had initially tried using VirtualBox to mimic my customer’s setup, but the issue never happened using VirtualBox.

My VM setup (even though it also uses Windows Display scaling) did not exhibit the problem. VirtualBox is weird when it comes to scaling and resolutions.

Then I tried using an old T420 Lenovo laptop. This only has a ‘normal’ display and is set to use 100% scaling so I did not see the issue on thact machine either.

It’s only when I pulled out my MS Surface 4 which has a HiDPI display and scaling set to 200% that I could duplicate the issue my end.

I created a ‘dummy’ extension using CEPSparker

This extension is a simple ‘HelloWorld’ panel with size set to 600×600.

The ZXP file can be downloaded here:
https://rorohiko.com/downloads/ID2020PanelSizeBug/ID2020PanelSizeBug.1.0.0.zxp

The source code archive for this ZXP can be downloaded here:
https://rorohiko.com/downloads/ID2020PanelSizeBug/ID2020PanelSizeBug.zip

If I have scaling at 100%, it looks like this when I open the dummy panel:

With the scaling at 200%, I open the panel:

And then I get this:

The panel is wider than my whole screen.