By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

L3 Workshop - Execution - Testing - Online - TRANREPT - Step - STEP05R - SORT - Include COND does not seem to work as expected.

0

Hi,

In L3 workshop, Online testing where in we generate transaction report thru JOB - TRANREPT which is triggered from online screen (Transaction CC00 - option 09). One of the steps in the JOB has SORT with INCLUDE COND in it. This particular step is returning with 0 SORT output records.

The step statements are as follows:

def stepSTEP05R(Object shell, Map params, Map programResults){ shell.with { if (checkValidProgramResults(programResults)) { return execStep("STEP05R", "SORT", programResults, { mpr .withFileConfigurations(new FileConfigurationUtils() .withJobContext(jobContext) .gdgSupport("SORTIN") .name("AWS.M2.CARDDEMO.TRANSACT.BKUP").ownerPath(".").relativeGeneration(1).storageProvider("filesystem") .disposition("SHR") .build() .fileSystem("SYMNAMES") .stream("", getEncoding()) .build() .fileSystem("SYSIN") .stream(""" SORT FIELDS=(263,16,ZD,A) ** INCLUDE COND=(305,10,CH,GE,C'${startDate}',AND, 305,10,CH,LE,C'${endDate}')""", getEncoding())** .build() .systemOut("SYSOUT") .output("") .build() .gdgSupport("SORTOUT") .name("AWS.M2.CARDDEMO.TRANSACT.DALY").ownerPath(".").relativeGeneration(1).storageProvider("filesystem") .disposition("NEW") .normalTermination("CATLG") .abnormalTermination("DELETE") .dcbParameters(".SORTIN") .recordSize(350) .build() .getFileConfigurations()) .withParameters(params) .runProgram("SORT") }) } } }

Output of the Step is as follows:

2024-11-25_16-02-38-358 | [STEP] STEP05R - Started 2024-11-25T15:02:38.367Z WARN 2940 --- [ Thread-16] c.n.b.g.i.s.c.b.FileConfigurationBuilder : Update of GDGSUpportConfiguration from catalog is not implemented. 2024-11-25T15:02:38.418Z INFO 2940 --- [ Thread-16] c.n.b.g.r.c.i.ExecutionControllerImpl : Executing program: SORT 2024-11-25T15:02:38.418Z INFO 2940 --- [ Thread-16] c.n.b.g.rt.db.stats.DatabaseStatistics : DATABASE STATISTIC DISABLED 2024-11-25T15:02:38.428Z INFO 2940 --- [ Thread-16] c.n.b.g.io.support.basic.BuilderHelper : Using path =[.\AWS.M2.CARDDEMO.TRANSACT.DALY.G0047V00] for FileSystem GDG named AWS.M2.CARDDEMO.TRANSACT.DALY generation 47 2024-11-25T15:02:38.428Z INFO 2940 --- [ Thread-16] c.n.b.g.r.job.support.DefaultJobContext : Stored [.\AWS.M2.CARDDEMO.TRANSACT.DALY.G0047V00] for key (AWS.M2.CARDDEMO.TRANSACT.DALY,1) 2024-11-25T15:02:38.429Z INFO 2940 --- [ Thread-16] c.n.b.gapwalk.io.support.FileFactory : Created new file [D:\PhotonUser\Workshop\working-directory..\AWS.M2.CARDDEMO.TRANSACT.DALY.G0047V00] 2024-11-25T15:02:38.433Z INFO 2940 --- [ Thread-16] c.n.b.g.r.job.support.DefaultJobContext : Returning stored dataset path from job context for key (AWS.M2.CARDDEMO.TRANSACT.BKUP,1) 2024-11-25T15:02:38.433Z INFO 2940 --- [ Thread-16] c.n.b.g.io.support.basic.BuilderHelper : Retrieved path from jobContext : .\AWS.M2.CARDDEMO.TRANSACT.BKUP.G0109V00 2024-11-25T15:02:38.939Z INFO 2940 --- [ Thread-16] c.n.b.g.r.sort.service.AdaptableSort : External sort threshold is 2071130352 bytes 2024-11-25T15:02:38.946Z INFO 2940 --- [ Thread-16] c.n.b.g.r.job.support.DefaultJobContext : Returning stored dataset path from job context for key (AWS.M2.CARDDEMO.TRANSACT.BKUP,1) 2024-11-25T15:02:38.946Z INFO 2940 --- [ Thread-16] c.n.b.g.io.support.basic.BuilderHelper : Retrieved path from jobContext : .\AWS.M2.CARDDEMO.TRANSACT.BKUP.G0109V00 2024-11-25T15:02:38.948Z INFO 2940 --- [ Thread-16] c.n.b.gapwalk.io.utils.CommonDDUtils : Record length read from the catalog for GDG dataset AWS.M2.CARDDEMO.TRANSACT.BKUP Generation 109 : 350 2024-11-25T15:02:38.953Z INFO 2940 --- [ Thread-16] c.n.b.g.r.job.support.DefaultJobContext : Returning stored dataset path from job context for key (AWS.M2.CARDDEMO.TRANSACT.BKUP,1) 2024-11-25T15:02:38.953Z INFO 2940 --- [ Thread-16] c.n.b.g.io.support.basic.BuilderHelper : Retrieved path from jobContext : .\AWS.M2.CARDDEMO.TRANSACT.BKUP.G0109V00 2024-11-25T15:02:38.953Z INFO 2940 --- [ Thread-16] c.n.b.gapwalk.utility.common.FileUtils : .\AWS.M2.CARDDEMO.TRANSACT.BKUP.G0109V00 - Existing file! - no file creation. 2024-11-25T15:02:39.531Z INFO 2940 --- [ Thread-16] c.n.b.g.u.s.s.e.ExternalSortTasklet : fcOut:GDGSupportConfiguration [relativeGeneration=1, name=AWS.M2.CARDDEMO.TRANSACT.DALY, ownerPath=., cacheCompression=false, warmUp=true, storageProvider=filesystem, recordSize=350, jobContext=com.netfective.bluage.gapwalk.rt.job.support.DefaultJobContext@6c00f8b8, rdw=false, bdw=false, isFixed=false, isLF=false, isCRLF=false, isMicroFocusCompatible=false, order=LIFO] 2024-11-25T15:02:39.540Z INFO 2940 --- [ Thread-16] c.n.b.g.r.job.support.DefaultJobContext : Returning stored dataset path from job context for key (AWS.M2.CARDDEMO.TRANSACT.DALY,1) 2024-11-25T15:02:39.540Z INFO 2940 --- [ Thread-16] c.n.b.g.io.support.basic.BuilderHelper : Retrieved path from jobContext : .\AWS.M2.CARDDEMO.TRANSACT.DALY.G0047V00 2024-11-25T15:02:39.541Z INFO 2940 --- [ Thread-16] c.n.b.gapwalk.utility.common.FileUtils : .\AWS.M2.CARDDEMO.TRANSACT.DALY.G0047V00 - Existing file! - no file creation. 2024-11-25T15:02:39.543Z INFO 2940 --- [ Thread-16] c.n.b.g.r.job.support.DefaultJobContext : Returning stored dataset path from job context for key (AWS.M2.CARDDEMO.TRANSACT.DALY,1) 2024-11-25T15:02:39.543Z INFO 2940 --- [ Thread-16] c.n.b.g.io.support.basic.BuilderHelper : Retrieved path from jobContext : .\AWS.M2.CARDDEMO.TRANSACT.DALY.G0047V00 2024-11-25T15:02:39.543Z INFO 2940 --- [ Thread-16] c.n.b.gapwalk.utility.common.FileUtils : .\AWS.M2.CARDDEMO.TRANSACT.DALY.G0047V00 - Existing file! - no file creation. **2024-11-25T15:02:39.544Z INFO 2940 --- [ Thread-16] c.n.b.g.u.s.s.e.ExternalSortTasklet : Sort output file length = 0 bytes ** 2024-11-25T15:02:39.549Z INFO 2940 --- [ Thread-16] c.n.b.g.r.job.support.DefaultJobContext : Returning stored dataset path from job context for key (AWS.M2.CARDDEMO.TRANSACT.DALY,1) 2024-11-25T15:02:39.549Z INFO 2940 --- [ Thread-16] c.n.b.g.io.support.basic.BuilderHelper : Retrieved path from jobContext : .\AWS.M2.CARDDEMO.TRANSACT.DALY.G0047V00 2024-11-25T15:02:39.549Z INFO 2940 --- [ Thread-16] c.n.b.g.i.c.i.AbstractDatasetCatalogImpl : Register Dataset - GDG computed absolute generation 47 2024-11-25T15:02:39.549Z INFO 2940 --- [ Thread-16] c.n.b.g.i.c.i.AbstractDatasetCatalogImpl : Registering a new dataset file AWS.M2.CARDDEMO.TRANSACT.DALY 2024-11-25T15:02:39.552Z INFO 2940 --- [ Thread-16] c.n.b.g.r.job.support.DefaultJobContext : Returning stored dataset path from job context for key (AWS.M2.CARDDEMO.TRANSACT.DALY,1) 2024-11-25T15:02:39.552Z INFO 2940 --- [ Thread-16] c.n.b.g.io.support.basic.BuilderHelper : Retrieved path from jobContext : .\AWS.M2.CARDDEMO.TRANSACT.DALY.G0047V00 2024-11-25T15:02:39.556Z INFO 2940 --- [ Thread-16] c.n.b.g.r.job.support.DefaultJobContext : Returning stored dataset path from job context for key (AWS.M2.CARDDEMO.TRANSACT.DALY,1) 2024-11-25T15:02:39.556Z INFO 2940 --- [ Thread-16] c.n.b.g.io.support.basic.BuilderHelper : Retrieved path from jobContext : .\AWS.M2.CARDDEMO.TRANSACT.DALY.G0047V00 2024-11-25T15:02:39.556Z INFO 2940 --- [ Thread-16] b.g.r.i.s.i.DefaultGDGEventsQueueHandler : [309564935] - Stacked GDG Event :GDGEvent [provider=filesystem, fsPath=./.\AWS.M2.CARDDEMO.TRANSACT.DALY.G0047V00, gdgName=AWS.M2.CARDDEMO.TRANSACT.DALY, type=NEW_GENERATION, relativeGeneration=1, absoluteGeneration=47, ] for [Thread-16] 2024-11-25T15:02:39.556Z INFO 2940 --- [ Thread-16] c.n.b.gapwalk.io.support.FileFactory : FileUtils : stack new generation event 2024-11-25_16-02-39-560 | Program return code is 0 2024-11-25_16-02-39-562 | [STEP] STEP05R - Ended

If we look at the above output - **2024-11-25T15:02:39.544Z INFO 2940 --- [ Thread-16] c.n.b.g.u.s.s.e.ExternalSortTasklet : Sort output file length = 0 bytes **

The SORTOUT is empty. I tried SORT FIELDS=COPY without INCLUDE COND and it worked fine with sort output having records in it but when we use INCLUDE COND, the output file is empty.

I looked in here to see if anyone else is facing the same issue but I could not find any. Not sure what is missing or going wrong but the behavior of this sort step in not as expected. Could any please help me out in resolving this.

Thanks.

Topics
Migration & Modernization
Tags
AWS Blu Age
Language
English
Himaprasad
asked 13 days ago38 views
1 Answer
0

Based on the information provided, it appears that the INCLUDE COND statement in your SORT step is not working as expected, resulting in an empty output file. This issue could be due to a few potential reasons:

  1. Date format mismatch: The INCLUDE COND is comparing date fields, and there might be a mismatch between the format of the dates in your input file and the format you're using in the condition.

  2. Incorrect field positions: The condition is looking at positions 305-314 for the date comparison. Ensure that these positions actually contain the date values in your input records.

  3. Variable substitution: The condition uses ${startDate} and ${endDate} variables. Make sure these variables are correctly defined and contain the expected values before the SORT step is executed.

  4. Input file content: Verify that the input file (AWS.M2.CARDDEMO.TRANSACT.BKUP) contains records that should match the condition.

To troubleshoot this issue, you could try the following:

  1. Temporarily modify the INCLUDE COND to be less restrictive or remove it entirely to confirm that records are being read from the input file.

  2. Add a OUTFIL statement to your SORT step to create a small sample of the input records, which can help you verify the content and field positions.

  3. Double-check the values of ${startDate} and ${endDate} to ensure they are set correctly and in the expected format.

  4. If possible, examine the contents of the input file to confirm that there are records that should match your condition.

  5. Consider adding some debug logging before the SORT step to print out the values of ${startDate} and ${endDate}, ensuring they are what you expect.

If these steps don't resolve the issue, you may need to review the overall job configuration and data flow to ensure all components are working as expected. Additionally, consulting the specific documentation for the BluAge L3 workshop or reaching out to the workshop support channels might provide more targeted assistance for this particular environment.
Sources
AWS BluAge L3 Workshop - Execution - Test - Batch - COMBTRAN - Issue | AWS re:Post

profile picture
re:Post Agent
answered 13 days ago
  • Himaprasad
    12 days ago

    Response to potential reasons:

    1. INCLUDE COND does not work for any type of data starting for any position. Even tried the first 4 bytes as condition which did not work.
    2. Checked the field position in HxD and it is correct. Tried hardcoding too which also did not work.
    3. ${startDate} and ${endDate} - these are coming out correctly. Checked it as there is println for dates in the script.
    4. AWS.M2.CARDDEMO.TRANSACT.BKUP - Has data in it. All the data shows up if I used SORT FIELDS = COPY without INCLUDE.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content