Sorting by roa number

Project:RUcore SOLR Searching and Indexing
Version:8.x
Component:Code
Category:bug report
Priority:normal
Assigned:triggs
Status:closed
Description

On production I am sorting in descending order by ROA number. Solr field "roa_identifier_t" with ORDER_DESC.

The values returned are not accurate. As an example.

996 is appearing before 1002.

Comments

#1

Why is roa_identifier_t a text field? Do any of these identifiers have [A-Za-z] characters that would prevent creating roa_identifier_i ?

#2

I thought you might know why it wasn't being indexed as an integer. I think they are all meant to be integers. Rhonda would know for certain and you should probably consult with her.

#3

Assigned to:triggs» chadmills
Status:active» test

I created a new roa_identifier_i field and reindexed the ROA collection without incident. You could use this instead and it should sort correctly. Not sure how the other got created as _t. I didn't do that one as I remember.

#4

Status:test» fixed

Looking at the past issues these two came up.

<a href="https://software.libraries.rutgers.edu/node/2527" title="https://software.libraries.rutgers.edu/node/2527">https://software.libraries.rutgers.edu/node/2527</a>

<a href="https://software.libraries.rutgers.edu/node/2960" title="https://software.libraries.rutgers.edu/node/2960">https://software.libraries.rutgers.edu/node/2960</a>

2527 explains the origins of the "roa_identifier_t" field in the solr index. Item 2960 was never responded to, but is a duplicate of this issue filed in January 2015. I will mark 2960 as a duplicate of this issue, but it should have been addressed during 7.6 dev or some other time.

I tested this and it is now working as expected.

Thank you.

#5

Status:fixed» closed

#6

There is now a usable roa_identifier_f field on rep-prod.

#7

Version:7.6» 8.x
Assigned to:chadmills» triggs
Status:closed» active

I made the configuration change on production and this isn't working. Sorting using the roa_identifier_f ascending and descending gives me the same results and they are not valid. Here are the first 10 ROA numbers returned.

1) 449
2) 892.7
3) 892.4
4) 892.2
5) 892.3
6) 892.1
7) 892.6
8) 892.5
9) 436
10) 474

#8

I looked into Solr float sort issues and didn't really find much complaining. They are supposed to sort numerically. On rep-test, I was able to generate this, which looks right, on the command line:
triggs@rep-devel:/mellon/htdocs/dlr/EDIT> lynx -source "http://rep-test.libraries.rutgers.edu:8983/solr/select/?q=roa_identifier_f:*+AND+relation:rucore00000000252&start=0&rows=10&indent=on&sort=roa_identifier_f%20desc&fl=id,roa_identifier_f"
<?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">21</int>
<lst name="params">
<str name="sort">roa_identifier_f desc</str>
<str name="fl">id,roa_identifier_f</str>
<str name="indent">on</str>
<str name="start">0</str>
<str name="q">roa_identifier_f:* AND relation:rucore00000000252</str>
<str name="rows">10</str>
</lst>
</lst>
<result name="response" numFound="8" start="0">
<doc>
<str name="id">rutgers-lib:204111</str>
<float name="roa_identifier_f">892.5</float>
</doc>
<doc>
<str name="id">rutgers-lib:204110</str>
<float name="roa_identifier_f">892.4</float>
</doc>
<doc>
<str name="id">rutgers-lib:204112</str>
<float name="roa_identifier_f">892.3</float>
</doc>
<doc>
<str name="id">rutgers-lib:204114</str>
<float name="roa_identifier_f">892.2</float>
</doc>
<doc>
<str name="id">rutgers-lib:204115</str>
<float name="roa_identifier_f">892.1</float>
</doc>
<doc>
<str name="id">rutgers-lib:204113</str>
<float name="roa_identifier_f">892.0</float>
</doc>
<doc>
<str name="id">rutgers-lib:202392</str>
<float name="roa_identifier_f">879.0</float>
</doc>
<doc>
<str name="id">rutgers-lib:202393</str>
<float name="roa_identifier_f">878.1</float>
</doc>
</result>
</response>

#9

I think I found the issue. I don;t think all of the ROA resources on the production system have a 'roa_identifier_f' index value. An example is: rutgers-lib:41065 on production.

<?xml version="1.0" encoding="UTF-8"?>
<response>

<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">1</int>
<lst name="params">
<str name="indent">on</str>
<str name="wt">xml</str>
<str name="q">id:rutgers-lib\:41065</str>
<str name="version">2.2</str>
</lst>
</lst>
<result name="response" numFound="1" start="0">
<doc>
<arr name="abstract"><str>I argue that the shift of explanatory burden from the generator to the evaluator in OT syntax – together with the difficulties that arise when we try to formulate a working theory of the interfaces of syntax – leads to a number of assumptions about syntactic structures in OT which are quite different from those typical of minimalist syntax: formal features, as driving forces behind syntactic movement, are useless, and derivational and representational economy are problematic for both empirical and conceptual reasons. The notion of markedness, central in Optimality Theory, is not fully compatible with the idea of synactic economy. Even more so, seemingly obvious cases of blocking by structural economy do not seem to result from grammar proper, but reflect (economical) aspects of language use.</str></arr>
<arr name="allmods"><str>

I argue that the shift of explanatory burden from the generator to the evaluator in OT syntax – together with the difficulties that arise when we try to formulate a working theory of the interfaces of syntax – leads to a number of assumptions about syntactic structures in OT which are quite different from those typical of minimalist syntax: formal features, as driving forces behind syntactic movement, are useless, and derivational and representational economy are problematic for both empirical and conceptual reasons. The notion of markedness, central in Optimality Theory, is not fully compatible with the idea of synactic economy. Even more so, seemingly obvious cases of blocking by structural economy do not seem to result from grammar proper, but reflect (economical) aspects of language use.
Text
892.5

English

application/pdf
38 p.

Author

Vogel
Ralf
Universität Potsdam

2006
Universitätsverlag Potsdam

Potsdam

Phonology

Syntax

OT syntax

Generator

Markedness (Linguistics)

Syntax-semantics interface

Syntax-phonology interface

Book Chapter
The definitive version of this paper is published in Linguistics in Potsdam 25 (2006).
Vogel, R. (2006). The Simple Generator. In H. Broekhuis & R. Vogel (Eds), Linguistics in Potsdam 25. Optimality Theory and Minimalism: A possible Convergence? Potsdam : Universitätsverlag Potsdam
ISBN: 9783939469544 (published book)
The work on this paper was partially supported by a grant for the research group “Conflicting Rules in Language and Cognition”, project A3, “Minimality in Optimality Theory”, FOR-375/A3.

The Simple Generator

Optimality theory and minimalism: a possible convergence?

9783939469544

Grammar, Comparative and general--Syntax

Rutgers Optimality Archive

rucore00000002165

NjNbRU

doi:10.7282/T3833PZK
</str></arr>
<arr name="author"><str>Vogel Ralf Author</str></arr>
<arr name="author_st"><str>Vogel, Ralf</str></arr>
<arr name="authorletter_st"><str>V</str></arr>
<arr name="availability"><str>Open</str></arr>
<arr name="collectionname"><str>Rutgers Optimality Archive</str></arr>
<arr name="contentmodel"><str>Document</str><str>Document</str></arr>
<arr name="datecreated"><str>2006</str></arr>
<int name="datecsort_i">20060101</int>
<str name="fedoraid">rutgers-lib_41065</str>
<arr name="genre"><str>Book Chapter</str></arr>
<str name="id">rutgers-lib:41065</str>
<arr name="identifier"><str>892.5</str><str>doi:10.7282/T3833PZK</str></arr>
<arr name="language"><str>English</str></arr>
<arr name="mint_txt"><str>Potsdam</str></arr>
<arr name="modsextent_txt"><str>38 p.</str></arr>
<arr name="modsgenrerulibfs_txt"><str>Book Chapter</str></arr>
<arr name="modsname"><str>Vogel Ralf Author</str></arr>
<arr name="modsnamerole_txt"><str>Author</str></arr>
<arr name="note"><str>The definitive version of this paper is published in Linguistics in Potsdam 25 (2006).</str><str>Vogel, R. (2006). The Simple Generator. In H. Broekhuis & R. Vogel (Eds), Linguistics in Potsdam 25. Optimality Theory and Minimalism: A possible Convergence? Potsdam : Universitätsverlag Potsdam</str><str>ISBN: 9783939469544 (published book)</str>
<str>The work on this paper was partially supported by a grant for the research group “Conflicting Rules in Language and Cognition”, project A3, “Minimality in Optimality Theory”, FOR-375/A3.</str></arr>
<int name="numid_i">41065</int>
<date name="objcreatedate_dt">2013-09-18T18:55:13.917Z</date>
<date name="objlastmoddate_dt">2013-09-18T18:55:13.917Z</date>
<arr name="portalkey"><str>DLR</str><str>8Vd1Cs3Kf</str><str>root</str>
<str>6Iu8My4Gr</str><str>digitalcoll</str></arr>
<arr name="publisher"><str>Universitätsverlag Potsdam</str></arr>
<arr name="relation"><str>rucore00000002165</str></arr>
<arr name="rights"><str>Copyright for scholarly resources published in RUcore is retained by the copyright holder. By virtue of its appearance in this open access medium, you are free to use this resource, with proper attribution, in educational and other non-commercial settings. Other uses, such as reproduction or republication, may require the permission of the copyright holder.Copyright protectedOpenCopyright holderVogelRalf</str></arr>
<str name="roa_identifier_t">892.5</str>
<str name="sortdate">sd:20060101</str>
<date name="sortdate_dt">2006-01-01T00:00:00Z</date>
<int name="sortdate_i">20060101</int>
<str name="sorttitle">simple generator</str>
<arr name="title"><str>The Simple Generator</str></arr>
<arr name="titleletter_st"><str>S</str></arr>
<arr name="topic"><str>Phonology</str><str>Syntax</str><str>OT syntax</str>
<str>Generator</str><str>Markedness (Linguistics)</str><str>Syntax-semantics interface</str><str>Syntax-phonology interface</str>
<str>Grammar, Comparative and general--Syntax</str></arr>
<arr name="topic_txt"><str>Phonology</str><str>Syntax</str><str>OT syntax</str>
<str>Generator</str><str>Markedness (Linguistics)</str><str>Syntax-semantics interface</str><str>Syntax-phonology interface</str>
<str>Grammar, Comparative and general--Syntax</str></arr>
<arr name="topicf"><str>Phonology</str><str>Syntax</str><str>OT syntax</str>
<str>Generator</str><str>Markedness (Linguistics)</str><str>Syntax-semantics interface</str><str>Syntax-phonology interface</str>
<str>Grammar, Comparative and general--Syntax</str></arr>
<arr name="type"><str>Text</str></arr>
</doc>
</result>
</response>

#10

The _f version is a clone of the other _t. Is there anything that prevented the object in quetion from being successfully reindexed? It looks like it to me just now for 41065, though the XML debug displays OK. Unfortunately the only actual indexing return I see is:
<responses><response actiontype="add"><status>Failed</status><message>Error with add action for rutgers-lib:41065...</message></response></responses>

#11

Status:active» closed

A solr configuration change on production was made that aligned it to test/dev. All is working as expected now.

Back to top