Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AIX: v4x-staging - object factory test failures #9825

Closed
mhdawson opened this issue Nov 28, 2016 · 5 comments
Closed

AIX: v4x-staging - object factory test failures #9825

mhdawson opened this issue Nov 28, 2016 · 5 comments
Labels
addons Issues and PRs related to native addons. aix Issues and PRs related to the AIX platform.

Comments

@mhdawson
Copy link
Member

  • Version: v4.x-staging
  • Platform: AIX
  • Subsystem: addons

Noticed what may be intermittent failures in addons tests in v4.x-staging stream:

https://ci.nodejs.org/job/node-test-commit-aix/nodes=aix61-ppc64/2113/console

not ok 1027 addons/02_object_factory/test
  ---
  duration_ms: 0.184
  severity: fail
  stack: |-
    module.js:327
        throw err;
        ^
    
    Error: Cannot find module './build/Release/addon'
        at Function.Module._resolveFilename (module.js:325:15)
        at Function.Module._load (module.js:276:25)
        at Module.require (module.js:353:17)
        at require (internal/module.js:12:17)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/test/addons/02_object_factory/test.js:4:15)
        at Module._compile (module.js:409:26)
        at Object.Module._extensions..js (module.js:416:10)
        at Module.load (module.js:343:32)
        at Function.Module._load (module.js:300:12)
        at Function.Module.runMain (module.js:441:10)
  ...

and earlier build:
https://ci.nodejs.org/job/node-test-commit-aix/nodes=aix61-ppc64/2111/

/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/test.tap:
not ok 1027 addons/02_object_factory/test
not ok 1028 addons/03_function_factory/test
not ok 1029 addons/04_wrapping_c_objects/test
not ok 1030 addons/05_factory_of_wrapped_objects/test
not ok 1031 addons/06_passing_wrapped_objects_around/test
not ok 1032 addons/07_atexit_hooks/test

not ok 1027 addons/02_object_factory/test
  ---
  duration_ms: 0.181
  severity: fail
  stack: |-
    module.js:327
        throw err;
        ^
    
    Error: Cannot find module './build/Release/addon'
        at Function.Module._resolveFilename (module.js:325:15)
        at Function.Module._load (module.js:276:25)
        at Module.require (module.js:353:17)
        at require (internal/module.js:12:17)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/test/addons/02_object_factory/test.js:4:15)
        at Module._compile (module.js:409:26)
        at Object.Module._extensions..js (module.js:416:10)
        at Module.load (module.js:343:32)
        at Function.Module._load (module.js:300:12)
        at Function.Module.runMain (module.js:441:10)
  ...
not ok 1028 addons/03_function_factory/test
  ---
  duration_ms: 0.186
  severity: fail
  stack: |-
    module.js:327
        throw err;
        ^
    
    Error: Cannot find module './build/Release/addon'
        at Function.Module._resolveFilename (module.js:325:15)
        at Function.Module._load (module.js:276:25)
        at Module.require (module.js:353:17)
        at require (internal/module.js:12:17)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/test/addons/03_function_factory/test.js:4:15)
        at Module._compile (module.js:409:26)
        at Object.Module._extensions..js (module.js:416:10)
        at Module.load (module.js:343:32)
        at Function.Module._load (module.js:300:12)
        at Function.Module.runMain (module.js:441:10)
  ...
not ok 1029 addons/04_wrapping_c_objects/test
  ---
  duration_ms: 0.173
  severity: fail
  stack: |-
    module.js:327
        throw err;
        ^
    
    Error: Cannot find module './build/Release/addon'
        at Function.Module._resolveFilename (module.js:325:15)
        at Function.Module._load (module.js:276:25)
        at Module.require (module.js:353:17)
        at require (internal/module.js:12:17)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/test/addons/04_wrapping_c_objects/test.js:4:15)
        at Module._compile (module.js:409:26)
        at Object.Module._extensions..js (module.js:416:10)
        at Module.load (module.js:343:32)
        at Function.Module._load (module.js:300:12)
        at Function.Module.runMain (module.js:441:10)
  ...
not ok 1030 addons/05_factory_of_wrapped_objects/test
  ---
  duration_ms: 0.189
  severity: fail
  stack: |-
    module.js:327
        throw err;
        ^
    
    Error: Cannot find module './build/Release/addon'
        at Function.Module._resolveFilename (module.js:325:15)
        at Function.Module._load (module.js:276:25)
        at Module.require (module.js:353:17)
        at require (internal/module.js:12:17)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/test/addons/05_factory_of_wrapped_objects/test.js:4:15)
        at Module._compile (module.js:409:26)
        at Object.Module._extensions..js (module.js:416:10)
        at Module.load (module.js:343:32)
        at Function.Module._load (module.js:300:12)
        at Function.Module.runMain (module.js:441:10)
  ...
not ok 1031 addons/06_passing_wrapped_objects_around/test
  ---
  duration_ms: 0.188
  severity: fail
  stack: |-
    module.js:327
        throw err;
        ^
    
    Error: Cannot find module './build/Release/addon'
        at Function.Module._resolveFilename (module.js:325:15)
        at Function.Module._load (module.js:276:25)
        at Module.require (module.js:353:17)
        at require (internal/module.js:12:17)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/test/addons/06_passing_wrapped_objects_around/test.js:4:22)
        at Module._compile (module.js:409:26)
        at Object.Module._extensions..js (module.js:416:10)
        at Module.load (module.js:343:32)
        at Function.Module._load (module.js:300:12)
        at Function.Module.runMain (module.js:441:10)
  ...
not ok 1032 addons/07_atexit_hooks/test
  ---
  duration_ms: 0.170
  severity: fail
  stack: |-
    module.js:327
        throw err;
        ^
    
    Error: Cannot find module './build/Release/addon'
        at Function.Module._resolveFilename (module.js:325:15)
        at Function.Module._load (module.js:276:25)
        at Module.require (module.js:353:17)
        at require (internal/module.js:12:17)
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/test/addons/07_atexit_hooks/test.js:4:15)
        at Module._compile (module.js:409:26)
        at Object.Module._extensions..js (module.js:416:10)
        at Module.load (module.js:343:32)
        at Function.Module._load (module.js:300:12)
        at Function.Module.runMain (module.js:441:10)
  ...
@mscdex mscdex added addons Issues and PRs related to native addons. aix Issues and PRs related to the AIX platform. v4.x labels Nov 28, 2016
@sxa
Copy link
Member

sxa commented Nov 28, 2016

I haven't been able to replicate this running the run-ci target the v4.x-staging branch on my own AIX6.1 system as yet. The logs in question do have:

/bin/sh[2]: ./node: not found.

Repeated a number of times equivalent to the number of failing tests which suggests it possible we've got an instance of the issue from #8285 - although that fix seems to be in the v4.x codebase.

@sxa
Copy link
Member

sxa commented Nov 28, 2016

There are two invocations of "ln -fs out/Release/node node" in the log - one of them is where I'd expect (just after the final link of the node binary) and the second is happening just before the "not found" errors are occurring. Looking to find out where that's coming from as it seems likely that the second one is what's causing the problem ...

@sxa
Copy link
Member

sxa commented Nov 28, 2016

The second one is occurring because the test-ci triggers the build-addons target which triggers ${NODE_EXE}

${NODE_EXE} is listed as a .PHONY target which is causing it to get triggered regardless of whether it needs to, which makes the "ln -fs out/Release/node node" get triggered again. I suspect that stopping this would resolve it. We could add a "[ ! -r node] && ln -s out/Release/node node", or remove the -f and add || true to the end of the line. Either would stop it trying to override the forced removal/recreation which I'm guessing (without being able to replicate it) is causing a problem with the way the file system operations are happening on AIX - I'm suspecting that the delete/recreate is "more atomic" on other operating systems.

(NOTE: The condition appears to still be in place on master so I'm not sure why we're only just hitting it now)

@sxa sxa mentioned this issue Nov 28, 2016
Closed
2 tasks
sxa pushed a commit to sxa/node that referenced this issue Nov 29, 2016
build: Don't regenerate node symlink
f69461c 
The node -> out/*/node symlink  is getting recreated in parallel with
other targets in the makefile which require it (e.g.  test-ci) and
this seems to be causing a race condition which is showing up on AIX

Ref: nodejs#9825
@mhdawson mhdawson reopened this Dec 5, 2016
@mhdawson
Copy link
Member Author

mhdawson commented Dec 5, 2016

Reopening until we land the backport on v4.x

addaleax pushed a commit that referenced this issue Dec 7, 2016
@addaleax
build: Don't regenerate node symlink
9d04152 
The node -> out/*/node symlink is getting recreated in parallel with
other targets in the makefile which require it (e.g. test-ci) and
this seems to be causing a race condition which is showing up on AIX

Fixes: #9825
PR-URL: #9827
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-by: Michael Dawson <michael_dawson@ca.ibm.com>
jmdarling pushed a commit to jmdarling/node that referenced this issue Dec 8, 2016
build: Don't regenerate node symlink
52a1eb1 
The node -> out/*/node symlink is getting recreated in parallel with
other targets in the makefile which require it (e.g. test-ci) and
this seems to be causing a race condition which is showing up on AIX

Fixes: nodejs#9825
PR-URL: nodejs#9827
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-by: Michael Dawson <michael_dawson@ca.ibm.com>
MylesBorins pushed a commit that referenced this issue Mar 7, 2017
@MylesBorins
build: Don't regenerate node symlink
32e4609 
The node -> out/*/node symlink is getting recreated in parallel with
other targets in the makefile which require it (e.g. test-ci) and
this seems to be causing a race condition which is showing up on AIX

Fixes: #9825
PR-URL: #9827
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-by: Michael Dawson <michael_dawson@ca.ibm.com>
MylesBorins pushed a commit that referenced this issue Mar 7, 2017
@MylesBorins
build: Don't regenerate node symlink
756cc91 
The node -> out/*/node symlink is getting recreated in parallel with
other targets in the makefile which require it (e.g. test-ci) and
this seems to be causing a race condition which is showing up on AIX

Fixes: #9825
PR-URL: #9827
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-by: Michael Dawson <michael_dawson@ca.ibm.com>
MylesBorins pushed a commit that referenced this issue Mar 9, 2017
@MylesBorins
build: Don't regenerate node symlink
f44c0a5 
The node -> out/*/node symlink is getting recreated in parallel with
other targets in the makefile which require it (e.g. test-ci) and
this seems to be causing a race condition which is showing up on AIX

Fixes: #9825
PR-URL: #9827
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-by: Michael Dawson <michael_dawson@ca.ibm.com>
MylesBorins pushed a commit that referenced this issue Mar 9, 2017
@MylesBorins
build: Don't regenerate node symlink
87db4f7 
The node -> out/*/node symlink is getting recreated in parallel with
other targets in the makefile which require it (e.g. test-ci) and
this seems to be causing a race condition which is showing up on AIX

Fixes: #9825
PR-URL: #9827
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-by: Michael Dawson <michael_dawson@ca.ibm.com>
@gibfahn
Copy link
Member

gibfahn commented Mar 29, 2017

Backport landed in Node 4.8.1: #11760

@gibfahn gibfahn closed this as completed Mar 29, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
addons Issues and PRs related to native addons. aix Issues and PRs related to the AIX platform.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mscdex @sxa @mhdawson @gibfahn