Skip to content

Event source length limited by the way it is passed to the runtime #29

Description

@ed-sparkes

There seems to be some hard limit on the event source that this runtime will process.

I am providing a large event source, a json stucture with one field containing an array of 20,000 observations (each observation has 3 fields).

I am receiving the following runtime error

START RequestId: da723778-9f4e-4198-a889-567b0ca8d790 Version: $LATEST
/opt/bootstrap: line 13: /opt/R/bin/Rscript: Argument list too long
END RequestId: da723778-9f4e-4198-a889-567b0ca8d790
REPORT RequestId: da723778-9f4e-4198-a889-567b0ca8d790 Duration: 112.00 ms Billed Duration: 200 ms Memory Size: 1028 MB Max Memory Used: 285 MB
RequestId: da723778-9f4e-4198-a889-567b0ca8d790 Error: Runtime exited with error: exit status 1
Runtime.ExitError

Event source json data attached as txt (would not allow me to upload json file)

The R function is not particularly relevant as its not getting that far, but included below anyway. Its dependent on PlackettLuce package which i have loaded in a separate layer. But this should be reproducable with a very basic Lambda function as it appears that the error is an the runtime invocation before the handler is event called.

library(PlackettLuce)

handler <- function(cjs) {
    #NOW USE FUNCTIONS FROM THE PLACKETT-LUCE PACKAGE TO DO THE WORK
    cjranks=rankings(cjs,id=1,item=2,rank=3)#put into bespoke format for PlackettLuce package
    con=connectivity(cjranks)#check connectivity of design (note that any individual object winning/losing all comparisons may be deemd an issue when it isn't really)
    mod <- PlackettLuce(cjranks)#fit Plackett-Luce model

    #more detailed connectivity (how many other objects in each objects cluster)
    connect.oth=sapply(1:length(con$membership),function(i) con$csize[con$membership[i]]-1)

    #READ THE COEFFICIENTS AND STANDARD ERRORS
    estmeas=coef(mod,ref=NULL)
    estse=sqrt(diag(vcov(mod, ref=NULL)))
    measures.data.frame=data.frame(id=names(estmeas)
        ,measure=estmeas
        ,se=estse
        ,connectivity.no=con$no
        ,connect.oth=connect.oth)

    #VIEW THE OUTPUT
    measures.data.frame

}

testdata.txt

This looks releavant (https://stackoverflow.com/questions/24658534/usr-bin-rscript-argument-list-too-long)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions