ERROR: SQL-MR function TEXT_PARSER failed: Loading stop word file /stopwords_en.txt failed.

Aster
Enthusiast

ERROR: SQL-MR function TEXT_PARSER failed: Loading stop word file /stopwords_en.txt failed.

Hi guys,

I am new to Aster. Only like 3 weeks into it.

I am using TEXT_Parser analytic function. I want to use my own stop word file.

I have loaded the file from my local into AFS using this command:

 \afs -copyFromLocal ./Test/stopwords_en.txt adfs:///

The file has successfully copied into the AFS.

Now I want to use this file in Text_Parser function:

SELECT Count(*) FROM text_parser

(

ON test2

TEXT_COLUMN('text1')

CASE_INSENSITIVE('true')

STEMMING('true')

TOTAL('true')

ACCUMULATE('sid')

REMOVE_STOP_WORDS('true')

LIST_POSITIONS('false')

OUTPUT_BY_WORD('true')

STOP_WORDS('adfs:///stopwords_en.txt')

);

When the run the above function, I get the following error:

Executed as Single statement.  Failed [34 : HY000] [AsterData][ASTERJDBCDSII](34) ERROR: SQL-MR function TEXT_PARSER failed: Loading stop word file adfs:/stopwords_en.txt failed. () 

Guys, please can u tell me how to specify the file path in the function.

I have tried many combinations and almost all of them have failed.

Tags (1)
2 REPLIES
Enthusiast

Re: ERROR: SQL-MR function TEXT_PARSER failed: Loading stop word file /stopwords_en.txt failed.

Well I found the solution -

I did not have to store the stop word file in AFS instead I needed to install it.

Steps to install :

1. Login to the database

2. Use the following command -

\install file [installed_filename]


Installs the file or function called file. The argument,

file, is the path name of the file relative to the

directory where ACT is running.

Optionally, you can give the file or function an

installed_filename alias. Aliases are provided as a

convenience that’s mainly useful for renaming helper

files you install. Using an alias for an SQL-MapReduce

function can be confusing, so we don’t recommend

doing it.

If no installed_filename is specified, the file’s

name will be used as its name in Aster Database. Keep

in mind that, when you call an SQL-MapReduce

function in your queries, you drop its filename suffix.

If the file or function does have an

installed_filename, then all calls to it from other

functions or from queries must use its

installed_filename.
New Member

Re: ERROR: SQL-MR function TEXT_PARSER failed: Loading stop word file /stopwords_en.txt failed.

Hi. I am running into the same issue. Unfortunately the \install function does not seem to work for me. My file is stored in a CSV and I even loaded my Stopwords file into a table in my schema, but that does not seem to work either. Do you have any sample code that you were able to run and are willing to share? Thank you! Please see code below:

 

SELECT    *
FROM    TEXT_PARSER (
        ON IAD.LAH_PRISM_PLANRULES_SAMPLE6
        TEXT_COLUMN('PLAN_RULES_TRIMFIRSTWORDS')
        TOTAL('TRUE')
        OUTPUT_BY_WORD('FALSE')
        PUNCTUATION('[\\\[.,?\!:;~()\\\]]')
        ACCUMULATE('ID_NO', 'PLAN_RULES', 'PLAN_RULES_TRIMFIRSTWORDS')
        REMOVE_STOP_WORDS('TRUE')
        STOP_WORDS('C:/Users/holdenl/Desktop/stopwords.csv')
        )
;