Eggplant AWC Testing

Skill eggplant-awc-testing. Drive Active Workspace (or any web app on a Windows SUT) with Eggplant Functional 26.2.2 headlessly - the runscript CLI, the RDP connection recipe, OCR/text matching that lets a suite be authored with no GUI and no pre-captured image assets, and the seven failure modes that make a run report success while proving nothing. Covers the credential handling that keeps a password out of the logs (Eggplant writes TypeText content to LogFile.txt by default), the blank-framebuffer race after Connect, SSO sessions that silently drop mid-test, and why OCR flakiness must be designed around rather than retried. Use for any Eggplant Functional work, any "my UI test is flaky" investigation, and before writing UI tests against Teamcenter.

EXERCISED 2026-08-25 against Active Workspace on the local TC2606 VM. Five tests, 21 assertions, green twice consecutively. Everything below came back from a real run; nothing here is derived from documentation alone. Where a claim is doc-only it says so.

The shape that works

Eggplant runs on the host, opens an RDP session into the VM, and drives a browser on that remote desktop. The application identity and the Windows identity are different accounts and must not be conflated.

host (Eggplant 26.2.2)  --RDP 3389-->  Windows SUT (as Administrator)
                                            |
                                        Chrome --> https://<awc-host>/
                                            |
                                   Keycloak SSO --> AWC signed in as <app user>

runscript.bat runs a suite with no GUI:

& "C:\Program Files\Eggplant\runscript.bat" ".\AWC.suite\Scripts\run_all.script"

A .suite is a plain folder (Scripts/, Images/, Results/), so suites are authorable with ordinary file tools. Exit 0 = pass, 1 = fail. Each run writes Results/<script>/<timestamp>/ with LogFile.txt, .json and .xml.

★ LogFile.xml is genuine JUnit <testsuites> format. Verified with a negative control: a failing run reports failures=13 errors=11 where a passing one reports failures=0, so the file discriminates rather than merely existing.

⚠ But Eggplant emits one <testcase> per SCRIPT, not per assertion. A runner script that calls fourteen tests produces ONE testcase with tests=1. In CI that shows a single test and cannot say which one broke. To get per-test reporting, pass the tests to runscript as separate script arguments — each gets its own result directory and its own JUnit file. Do not promise "CI needs no adapter" without that qualification; the granularity is the catch.

Connecting

Connect serverID:"192.168.222.100", portNum:3389, type:"RDP", username:"Administrator", password:pw

type:"RDP" needs no extra install: Eggplant 26.x ships FreeRDP. VNC is the default type and is not required. RemoteScreenSize() confirms the session, Disconnect ends it.

⚠ An RDP logon fires the SUT's Startup folder. On a Teamcenter demo VM that can mean a service-restart script. Check what is in the all-users Startup folder before running a suite that reconnects repeatedly, and see tc-vm-operations.

The API, corrected against three wrong guesses

Each of these cost a round trip. They are what 26.2.2 actually has:

Want	Wrong	Right
environment variable	EnvironmentVariable("X"), the environment	env("X")
write to the log	LogMessage	log, plus LogError / LogWarning / LogSuccess
OCR a region	ReadText(EntireScreen)	ReadText((x1,y1),(x2,y2))
shell out	ShellCommand(...)	shell(...), but see below

☠ put produces NOTHING in a headless run. It reaches neither stdout nor the log file. A script full of put statements runs green and tells you absolutely nothing, which is the classic success-shaped silence. Use log.

⚠ the environment is not OS environment variables: it is Eggplant's own environment name and evaluates to a string like development. Reading a variable off it silently returns empty rather than erroring, so it looks like an unset variable rather than a wrong API.

⚠ shell() exists but returns empty on Windows, because the shellCommand global defaults to ShellExecute. Set the shellCommand to empty first if you need it. (DERIVED from the docs; the env() route removed the need to exercise this.)

Credentials: Eggplant writes your password into the log

☠☠ TypeText content is logged by default. Typing a password puts it in clear text in LogFile.txt and LogFile.json. This is not hypothetical: it happened on the first run of a login script here and the artifacts had to be scrubbed.

Click passwordField
set the scriptLogging to Silent
TypeText pw
set the scriptLogging to On

TypeEncodedText is the alternative and is never logged. scriptLogging levels are On / Actions / Off / Minimal / Silent.

⇒ After writing any script that types a secret, grep the resulting log before committing. The suppression is invisible when it works and invisible when you forget it.

Pass the secret in via env(), populated by the caller decrypting the DPAPI credential in memory. Not on the command line (visible in the process list) and not in a file.

$c = Import-Clixml "$env:USERPROFILE\.xcelerator\credentials\vm-admin.cred.xml"
$env:EGG_SUT_PW = $c.GetNetworkCredential().Password
# ... run ...
$env:EGG_SUT_PW = $null

★ Make a login helper attempt exactly once. Retrying against Keycloak risks locking the account, and a lockout on a shared demo tier is far more expensive than a failed test.

OCR/text matching, which is what makes headless authoring possible

Image-based matching needs pre-captured reference images, which you cannot easily produce without the GUI. Text matching needs nothing:

if ImageFound(text:"Favorites") then ...
WaitFor 12, (text:"Sign in")
Click (text:"Search")
put ReadText((0,140),(900,700)) into panelText

The licence used here reports OCR-enabled-unlimited usage, ABBYY engine.

Design around OCR's limits rather than fighting them:

• Small text is misread. 000427 reads as 0CC427; Inbox reads as Intox; AttOAttributeDefRevision reads as AttCAttri outeCef Revision. Never assert exact equality on small values. Assert contains on a region read.
• Large UI text is reliable. Panel headings, tab names and buttons matched every time.
• One region read beats N text searches. Reading a whole column in one ReadText gives the engine surrounding context, is much faster, and removes position dependence.
• ⚠ A failed text search costs ~66 seconds at default settings. Set the imageSearchTime (12 is a reasonable floor) and pass explicit timeouts to any assertion that is expected to fail.

★ Always ship a negative control. ImageFound(text:"ZZQXNOTONSCREEN") must not match. Without it you cannot tell a working matcher from one that returns true for everything. Run it in the same pass as the positive.

The seven ways a run reports success and proves nothing

Every one of these was hit while building the first suite.

1. Blank framebuffer after Connect. A fresh RDP session returns nothing for a second or two, so OCR reads 0 characters and the test fails for reasons unrelated to the application. Gate the connect helper on something cheap and real (WaitFor 30, (text:"<app banner>")) before returning. A test that happens to have a wait in it will pass while its neighbour fails, which reads as a product bug and is not.

2. The SSO session drops mid-suite. Typing a URL into the address bar bounced the browser to the Keycloak sign-in page. The next Click/TypeText then went into the login form, and the search string was submitted as a username. Make the navigation helper detect the sign-in page and authenticate, so a lapsed session self-heals instead of cascading into nonsense failures three tests later.

3. A login test that never logs in. If a session is already live, an "am I on the login page?" check takes the already-authenticated branch and the test reports PASS having tested nothing. It cannot distinguish a successful sign-in from a pre-existing one. ⇒ Sign out first (hit the IdP's /protocol/openid-connect/logout), assert you reached the sign-in page, and only then log in. Otherwise the most important test in the suite is decorative.

4. Asserting the wrong rectangle. An assertion read (900,140)-(1599,700) and failed. The browser window was ~1275 wide, so most of that region was desktop wallpaper. The failure was in the test, not the product. Cross-check any region against a screenshot before trusting either its pass or its fail.

5. Flakiness that looks like a defect. A nav-rail label passed at 03:07 and failed at 03:12 on an unchanged tier. Two causes, both about the rail: its top slot changes with navigation history (No previous location becomes Search after a search), shifting everything below it ~18px; and the labels are ~8px under notification badges. ⇒ Read the rail once as a column and assert on a majority (3 of 5), logging the raw OCR. A test that fails randomly trains people to ignore failures, which is worse than no test.

6. Naming a test for what you meant rather than what it does. Single-clicking a Favorites tile selects; double-clicking did not navigate either on this build. The test that was going to be "open the object page" became "the search result view renders the object's properties", because that is what it actually verifies. Rename the test rather than let the name overclaim.

7. A compile error kills the whole script silently-ish. One bad line means zero of the probes in that file run, and the log shows a single SenseTalk Compiler Exception rather than the results you were expecting. if the result then else ... is not valid SenseTalk; use if the result is false then .... When a probe file reports nothing at all, suspect syntax before behaviour.

Suite structure that held up

AWC.suite/Scripts/
  ConnectToAwc      connect + readiness gate, returns RemoteScreenSize()
  GoToAwcHome       navigate home, sign in if the session lapsed
  LoginToAwc        Keycloak sign-in, one attempt, password logging suppressed
  AssertText        WaitFor + LogSuccess/LogError + CaptureScreen on failure
  test_NN_*         one concern each, connect and disconnect independently
  run_all           dispatches each test with `do testName`, counts, asserts zero failures

Any .script in the suite is callable as a handler by its filename. To dispatch a name held in a variable use do testName (run is the shell command and will not do it).

⚠ AssertText captures a screenshot named after its label on failure. Sanitise the label first: Windows rejects : / \ | ; in filenames, and the capture then throws inside the failure path, replacing a clean assertion failure with a confusing file error.

The test that catches a faithful-but-wrong UI

Every assertion above compares the screen against a string written into the test. That proves the UI is self-consistent with whatever someone typed when the test was written. It cannot catch the UI rendering a stale or incorrect value, because the expected value is stale in exactly the same way. A screenshot-shaped suite sails straight past a wrong owner.

⇒ Fetch the expected values from the system of record immediately before the run, and assert the UI matches those. On Teamcenter that means SOA:

runner (PowerShell)
   -> oracle_fetch.py  (tc_client from tc-mcp; do NOT hand-roll the login)
   -> {item_id, object_name, owning_user, last_mod_date, revision_list}
   -> passed to the suite as ORACLE_* environment variables
   -> the SenseTalk test asserts the rendered screen contains them

EXERCISED 2026-08-25: API and UI agreed on all four compared properties, owning_user reading ed (ed) on both sides.

Three things that keep this honest:

• ★ If the oracle cannot be fetched, the test FAILS on the missing oracle. It must never skip, and must never pass by default. An absent oracle and a matching one cannot be allowed to look alike.
• ★ Compare the comparable part. The API returned 12-Aug-2026 23:13 and AWC rendered 12-Aug-2026. Comparing whole strings would fail for a formatting reason while reading like a data mismatch. A test that fails for the wrong reason misleads as badly as one that passes for the wrong reason.
• ★ Log which form matched when you tolerate OCR. Where a short numeric value needs a fallback form (000427 vs 0CC427), record that the match was tolerated rather than clean, so nobody reads a fuzzy pass as an exact one.

☠ Do not measure the run by parsing the runner's console output

Committed here on 2026-08-25 after writing the rest of this skill, which is the point: knowing the failure shape does not stop you producing it.

A stability check ran the suite three times and reported "STABLE: 3 consecutive runs, 0 failures". The verdict was true. The instrument was worthless:

$out  = .\Invoke-AwcSuite.ps1 -PerTest 2>&1 | Out-String   # <- captures NOTHING
$pass = @($out -split "`n" | Where-Object { $_ -match 'PASS' }).Count   # 0
$fail = @($out -split "`n" | Where-Object { $_ -match 'FAIL' }).Count   # 0
if ($fail -eq 0) { "STABLE" }                                          # cheerfully green

The runner reports through Write-Host, which writes to the information stream. 2>&1 redirects stderr, not stream 6, so $out was an empty string. Zero PASS matches and zero FAIL matches, and $fail -eq 0 is trivially true. Had all 42 tests failed it would have printed the identical summary. The per-run counts printed passed=0 failed=0 right next to the word STABLE and that inconsistency was the only visible tell.

⇒ Read the artifact, not the console. The JUnit XML is what Eggplant actually wrote and what CI would consume; its failures and errors attributes are the verdict. Console text is a rendering, and in PowerShell it may not even be capturable.

⇒ Then run the checker against known-bad input before believing it. Three controls settled it here in one pass: a known-good window (STABLE, exit 0), a window containing a run that genuinely failed earlier that night (FAILURES PRESENT, exit 1), and a deliberately corrupted XML (INCOMPLETE, exit 2, back to STABLE once restored). Without the second and third, the first proves nothing at all.

⇒ Three-valued outcomes, always. PASS / FAIL / UNREADABLE. A result that cannot be read must never fall into the pass bucket - which is exactly what caught a real gap here: a test killed mid-write by a shell timeout left a result directory with no XML, and a two-valued checker would have scored it green.

Writing to Teamcenter from a UI test, when Teamcenter cannot delete

On this tier Teamcenter never deletes (see never-delete-tc-objects). Every object a test creates is permanent, and that one fact should drive the whole design rather than being a footnote:

• One object per invocation. Not one per assertion, not a loop. A write test that creates five objects per run creates fifty by Friday.
• Name it so it is unmistakably test data and cannot collide: a fixed prefix plus a timestamp, e.g. EGGTEST-DOC-<yyyyMMdd-HHmmss>. They then sort together and a future session can tell them from real data at a glance.
• Keep a register. A record is the only cleanup available when removal is not. An unrecorded permanent object is exactly the one somebody later mistakes for real content.
• Give the entry point a -WhatIf that prints the name and creates nothing.

★★ Verify the create against the system of record, never against the UI that performed it. The screen can render an optimistic row, a cached list, or simply a panel that closed, and all three look like success. Two stages, and the second is the one that counts:

Eggplant test   -> drives the UI, reports what the screen showed
verify script   -> queries Teamcenter over SOA for the name  <- this decides

★ Four outcomes, not two: FOUND / NOT_FOUND / ERROR / AMBIGUOUS. ERROR must be distinct from NOT_FOUND, because a connection failure is not proof of absence, and neither is a pass. AMBIGUOUS matters more than it sounds: a name shared by several objects means the check cannot decide, and picking the first match would silently verify the wrong object.

★★★ Prove the verifier against controls BEFORE trusting it to judge an irreversible write. If the verifier is broken you either report a success that did not happen, or fail an object that now exists forever. Three controls settled it here in one pass: a unique existing object (FOUND), a name genuinely shared by six objects (AMBIGUOUS, refused to guess), and an absent name (NOT_FOUND). EXERCISED 2026-08-25, then used to confirm a real Document create.

⚠ SenseTalk trap hit on the first write attempt: initialising a variable with put empty into target and then assigning a screen location into it raises NSInvalidArgumentException: STGenericObject(instance) does not recognize setString:. Track the INDEX of the location in the list instead of the location itself.

⚠ And a UI trap worth knowing: a create panel's submit button is often not the only control with that label. EveryImageLocation(text:"Add") returned two, [[1006,236],[1217,818]] - the panel header and the actual button. Pick by position (lowest on screen), and log what you found, or a run will confidently click the wrong "Add".

☠☠ A single text match is not an anchor unless the text is unique

The most expensive recurring mistake in OCR-driven testing, hit THREE times in one session after the rest of this skill was already written. Each time the code looked obviously correct.

Anchor	What it actually matched	Consequence
"Teamcenter"	a DESKTOP ICON captioned "Teamcenter 2606"	reported the app was open against a Windows Server Manager desktop
"SIEMENS"	the VM's desktop wallpaper heading, not the app banner	clicked empty space; the user menu never opened
"Favorites"	the right panel, until the tier stopped rendering it	ONE absent panel aborted FIVE tests before any assertion ran

⇒ Two fixes, and which one you need depends on the case:

• Several matches exist, you want a specific one. Enumerate and select by POSITION. EveryImageLocation(text:"SIEMENS") returned [[182,62],[1548,104]]; the banner is the rightmost. The same technique picks a dialog's submit button when the panel header shares its label: EveryImageLocation(text:"Add") returned the header and the real button. Log how many matches you found. "found 1 of an expected 2" is the tell that something is covering the screen.
• You only need to know you are on the right page. Accept ANY of a set rather than demanding one. A navigation helper that waits on one panel turns a cosmetic change into a suite-wide outage, and none of the resulting failures name the real cause.

★ And prefer a probe that ONLY the running application can produce. "Any Category" (the search scope selector) and "Forgotten your password" (the sign-in page) cannot appear on a desktop, a shortcut caption or a wallpaper. "Teamcenter" can.

Bootstrapping the browser, and three ways it goes wrong

A suite that assumes a browser is already open is depending on someone's leftover desktop. It works until the session is rebuilt, then every test fails for a reason unrelated to the application. Open one yourself. The Run dialog (windowsKey, "r") is the most portable launcher: no desktop icon, no Start-menu layout, no install path.

★★ Make the recovery deterministic: close the browser entirely, then start exactly one. A helper that merely launches when it cannot see the app ACCUMULATES. Measured here: 36 chrome processes, 4.3 GB, and zero frontmost windows, because each new launch opened a tab in a window that was not on top, so the app was never visible, so it launched another. Nine consecutive tests then failed identically. A bootstrap that leaves residue eventually defeats itself.

⚠ But a force-kill has its own tail. taskkill /F makes the next Chrome launch show the "Restore pages? Chrome didn't shut down correctly" bubble, which sits over the TOP RIGHT of the Active Workspace banner and hides the wordmark and the user avatar. The cleanup created the obstruction that broke the next run. Launch with --hide-crash-restore-bubble.

Screen fractions are not layout fractions

ScreenRect-style regions expressed as fractions of the REMOTE SCREEN are only interchangeable with the application's layout while the browser window fills a constant share of that screen.

They diverged here: regions calibrated against a ~1275px window, then a maximised 1600px one. Seven tests failed while the application rendered perfectly, reading the right pages through the wrong rectangles. The logs proved it: the OCR text in them was correct and complete.

⇒ Maximise the window as part of connecting, so window == screen and the fractions mean what they claim. And when a test fails, read the OCR it logged before adjusting anything: if the text is right, the region is wrong, and no amount of tweaking assertions will help.

Testing requirements management

What is worth asserting, in increasing order of value:

1. The specification opens and is the right object. Cheap, proves little.
2. It actually contains requirements, asserted as real rows. This is the one that matters: a spec whose content is not carried opens fine, renders fine, passes every structural check, and shows an empty tree.
3. The requirement STATEMENT text renders, not merely titles. A tool that lists titles but cannot show bodies is unusable for requirements work, and 1 and 2 both still pass.
4. The configuration context is displayed (revision rule, variant rule, expansion rule). Requirements are configuration-dependent, so content shown without its applied configuration is an unlabelled answer.

★ Match requirement rows by TITLE, not id. OCR read AV-SR-002 as AV-SR-032 in a narrow Element column while reading the same value correctly in a wider one.

⚠ Several requirements capabilities are gated on infrastructure, so a test for one that is not configured fails like a product bug: Export to Excel and Baselining need AsyncService, import needs Dispatcher, the Word round trip needs the ReqMgmtWordToHTML/HTMLToWord translators, and the History tab needs REQ_CompareSpecificationHistory plus REQ_Microservice_Installed. Check what is actually enabled before testing it. ⚠ A stopped DispatcherClient is SILENT: async work queues forever with no error (see tc-vm-operations).

Prove the tier survived the test run

On a Teamcenter VM every RDP logon fires the guest's all-users Startup shortcut, which can point at a script that stops the tier and wipes its logs. Such scripts usually carry a guard that skips the teardown when the tier is healthy - and a guard that silently stops guarding looks exactly like one that works. Check by effect, in the runner, every run.

★ The signal is the OLDEST process start time, not the count. A tcserver pool churns on its own: counts of 8, 11, 12 and 11 were all observed across one night with nothing wrong. A teardown kills every one at once, so the discriminator is whether the oldest surviving tcserver predates the run.

$procs = @(Get-Process tcserver); ($procs | Sort-Object StartTime | Select-Object -First 1).StartTime

Report a three-valued outcome: PASS, TEARDOWN DETECTED, and COULD NOT RUN when the guest is unreachable. The third must be visibly distinct, or an unrunnable check reads as a green one - the same failure shape the guard itself has.

Related skills

tc-vm-operations (the VM, its Startup-folder behaviour, checkpoints), tc-sso-keycloak (the identity layer AWC sits behind), tc-awc-stylesheets (what is actually being rendered).

---

Generated from skills/eggplant-awc-testing/SKILL.md in the tc-automation-skills library, which is the canonical copy and also serves as the agent skill set for Teamcenter work.