Specifying Field Analyzers using Index Templates in Elasticsearch

I’ve been playing around with ElasticSearch recently, and I must say I’m quite impressed with it. However, I’ve had my fare share of poring over the internet to deal with specific challenges I’ve faced in making my application content easily searchable, and one was with preventing certain fields from being analyzed.

These fields are typically ids that should be stored as they are and not broken down. Some of these ids are UUIDs, and other applications specific ones with hyphen (“-“) separations. To enable it to match both possibilities of the whole id being supplied or some portions being supplied (which will required a wildcard search), these kinds of fields should not be analyzed but handled as is.

Since all my ids typically have 1 or more variations of the word “id” somewhere in the mix, the simple solution was to provide an index template that says not to analyze any such fields. The saviour was Elasticsearch’s index templates, and here is mine.

{
    "carewex_template": {
        "template": "*",
        "settings": {
            "index.number_of_shards": 2
        },
        "mappings": {
            "_default_": {
                "_all": {"enabled": false},
                "_source": {"compress": true},
                "dynamic_templates": [
                    {
                        "primarykey_template": {
                            "match": "*id",
                            "include_in_all": true,
                            "mapping": {"type": "string", "index": "not_analyzed"},
                            "match_mapping_type": "string"
                        }
                    },
                    {
                        "otherIdkey_template": {
                            "match": "*Id",
                            "include_in_all": true,
                            "mapping": {"type": "string", "index": "not_analyzed"},
                            "match_mapping_type": "string"
                        }
                    },
                    {
                        "thirdIdkey_template": {
                            "match": "*ID",
                            "include_in_all": true,
                            "mapping": {"type": "string", "index": "not_analyzed"},
                            "match_mapping_type": "string"
                        }
                    }
                ]
            }
        }
    }
}

Placing this file as my_template.json in the config/templates directory was all I needed. Whenever I indexed a document, all matching fields were prevented from being analyzed, and my problem was solved.

So, have fun with Elasticsearch.

Advertisements

Charts in JSF: OpenFaces, PrimeFaces and JSFLot

I’ve been playing around with a small application that needs to display results of data collection in a chart as a certain selection is made on the jsf page. So I set myself the task of looking around for libraries that could provide me this charting support. One of the important considerations was that this library had to be compatible with Richfaces, since that was my default jsf library until further notice. It was a pity though that Richfaces didn’t have one, because most of their components seem to be fully fleshed, and I don’t tend to need other components libraries unless they don’t have it.

Primefaces

I’d already heard and read about Primefaces, and the reviews were quite positive. It comes with quite an impressive set of components, and will definitely fill in the holes that Richfaces has left quite nicely, with all the cool items like Accordions, Carousels, Docks (for MacOS fans), an IdleMonitor, ImageCropper and the rest. The documentation was also quite detailed from PDF and html docs as well from forums, so I was bound to have a good time, or so I hoped. I “mavened” it and configured it alongside Richfaces, without any complaint. So, to the hacking went on. The chart components were quite may and as cool as Primefaces always tends to be, and it’s model was quite easy to work with. All I needed was to create a Map<String,Integer> containing text and data points for a pie chart. For my needs, which is a bar char, all that was needed was something like the ff from their own documentation:

public class BirthDisplayBean {

private List<Birth> births;

public ChartBean() {

births = new ArrayList<Birth>();

births.add(new Birth(2004, 120, 52));

births.add(new Birth(2005, 100, 60));

births.add(new Birth(2006, 44, 110));

births.add(new Birth(2007, 150, 135));

births.add(new Birth(2008, 125, 120));

}

public List<Birth> getBirths() {

return births;

}

}

and then on the page

<p:lineChart value="#{chartBean.births}" var="birth" xfield="#{birth.year}">

<p:chartSeries label="Boys" value="#{birth.boys}" />

<p:chartSeries label="Girls" value="#{birth.girls}" />

</p:lineChart>

Sweet! Simply using my own model and basic collections, I had my data all ready to go.

Just when I was getting ready to enjoy splattering my pages with charts all over, I came across a problem. The number of points on which data is collected in my application is flexible, therefore I do not know before hand the number of “series” that I have to display. Unfortunately, Primefaces assumes that I know them before hand, in which case all I needed was to specify each p:chartSeries with label and value. Oops, spanner in the works!! I tried to use a ui:repeat to force it to render a dynamic number of p:chartSeries, but that didn’t work. So, my honeymoon with Primefaces charts was ended abruptly. But Primefaces is still in my web application classpath, waiting for the next interesting component I might think of using which Richfaces does not have. I suspect that will definitely be sooner than later. Primefaces is way too cool to ignore.

JSFLot

My next search threw up an interesting result: JSFLot. It’s quite an interesting libray only focusing on charting, and relying mostly on JavaScript to render the chart and it’s content. It has support for pie, bar, and line charts, which would meet most application needs. It seemed not to have as yet a big community around it like the others, but it’s documentation was good enough to focus on what it does best – charting. I only wished there was a downloadable version of the documentation, so I could take my time with it at home when I’m offline. In the end I had to use Scrapbook to grab a few pages, but that was good enough. It indeed has a very small footprint, with a jar around 245k. It had it’s own datamodel that you had to stuff your results to display in, so in that sense it is intrusive on your codebase. However they are quite simple and intuitive. They are XYDataPoint (for x and y data points) , XYDataList (for a series of x and y data points, as well as other information concerning the series) and XYDataSetCollection (an aggregation of 1 or more series or XYDataLists). But nowadays, what other JSF component library doesn’t call for some small intrusion to get you going?

I began digging into it, and was getting some interesting results. The charts were quite clean and easy to label. But when I wanted to be a bit more dynamic and display different charts based on different selections from a Seam DataModelSelection, I noticed that it didn’t seem to refresh to show the changing data points from the different objects from which the data points were being displayed. I thought maybe it had to work only with full page refreshes, so I resorted to using the normal <h:commandLink/> to make sure that the whole page was refreshing and not doing any funky Richfaces ajax thingy. But no go. Seeing as I was spending too much time trying out all my Seam hacking skills, I rather decided to focus my energies on finding a different library that could fulfil my needs. Maybe I was being dumb and making some mistake somewhere, but time wasn’t on my side.

OpenFaces

Having had two heartbreaks, I went back to looking for a new JSF library love that could fulfil my need for dynamic series data, and interestingly 2 weeks ago TheServerSide had an article about OpenFaces. Hmm, not heard of them; let’s see what they’ve got. It turns out that they weren’t that bad after all. Documentation was in an html which comes bundled with the library download, and contains everything you need to know to use it. They had quite a sweet implementation of DataTable, and their implementation of sorting via headers of the columns was far cleaner and more “pimped up” than Richfaces Datatable, so I’ve switched my pages to use theirs, and I’m loving it. They also have a TreeTable, a cool way of using a table to display hierarchical structures, and a DayTable for showing scheduled events. All this and it sat quite well with Seam and Richfaces.

Here they had support for pie, bar and line charts, which though less than Primefaces’s plethora of charts, is more than enough for most purposes. Oh by the way, they could do dynamic series data quite well. All would have been rosy, except for the fact that I have to use their model to squeeze my data in. Well, it involved using 2 of OpenFaces model classes to contain my data, and coming from JSFLot’s 3 and from all the disappointments, I definitely could live with that. So like Primefaces, I could define my data points in a Map<String,Integer> structure, but unlike it, I’d put them in a PlainSeries, and then put all my PlainSeries in a PlainModel. Job done, we can all go have a beer now.

But then what will software development be like if you had technologies which thought of the developer’s every need and did them even before he could think he needed them? That will be utopia, but then I’m still on this earth. I realised that I couldn’t specify a color per data element, again because the number of data points I have to display is dynamic. I tried to use a property that will generate a random comma separated list of colours for each data point as a string property , but the tag could not resolve EL when it came to reading the “colors” property. In the end I had to hard code one colour for every element to save me from disgrace.

<o:barChartView labelsVisible="true" valueAxisLabel="No. of respondents" keyAxisLabel="Responses" colors="#800000"/>.

In fact, neither could the valueAxisLabel and the keyAxisLabel read from my locale files to determine the right text to show there. Who in this day and age still hard codes labels, when there is something call internationalization? OpenFaces, sit up!!! This is JSF, and here EL is king, not hardcoded text values.

In the meantime, I don’t have a choice. At least OpenFaces meets the real important requirement of showing charts from content which is dynamic and ajax driven. I hope OpenFaces will wake up and realise that their new lover requires some additional pampering, but I guess for now the relationship seems to be working. Who knows, if they do get better at the EL stuff, I just might actually consider moving from a relationship to a marriage.

One thing I’ve taken away from the experience though – JSF has come a long way, for me to be able to have Richfaces, Primefaces and OpenFaces in one application. And my application is not even JSF 2, where the vendors are supposed to have worked on better integration paths for the component libraries. I’m waiting for Seam 3 to be fully released, and then I’ll switch everything to CDI, JPA2 and JSF2 without sacrificing any of my PDFing, Excel-ling, SMPCing (Managed Persistence Context) and the like.

Book Review: Packt’s JBoss Richfaces 3.3

Summary

Very good book on what is arguably the most popular JSF component library. Covers the less well used but very important ajax features of Richfaces, including the use of ajax regions, containers as well as the plug-n-skin features.

Details

To all intents and purposes, this book is not for a beginner JSF developer, because it assumes familiarity with JSF 1.2. It starts off by delving into the history of the Richfaces component framework, which in eons past was just Ajax4jsf. It gives a short introduction (which is expounded later) on the ajax capabilities, the component set and skinability features of Richfaces for Rich Internet Application development.

It then sets off showing you step by step, how to create a simple JSF application that takes advantage of Richfaces as it’s JSF components library. It presents the seam-gen tool, a nifty tool for creating skeletons jsf projects based on the JBoss Seam project. Following the instructions on seam-gen enables you to create a quick prototype to start playing around with. This is then followed by instructions on how to manually create a project which is not Seam based, for fans of other frameworks which also integrate JSF, or for the pure JSF purists. This also includes configuring facelets support, and finally a look at how the IDEs provide support for development using the Richfaces component set.

The rest of the book, chapter by chapter, then focuses on using Richfaces components in building an interactive, intuitive and rich UI for your JSF application.

The following chapter delves into creating a JSF application using seam-gen (which I felt was repetitive, given that they’d actually created one in the preceding chapter), with additional parameters for defining the persistence model (entities) that were to be used in the application. It is worth noting from here forward almost all the examples now make use of the Seam framework’s programming model, something which could be a problem for those who are not Seam adherents. Also, with the stated assumption that this book is targeted at people with a basic knowledge of JSF, there didn’t seem to be the need to go over the basics of templating with Facelets and defining managed beans in a faces-config.xml file.

The next two chapters then introduce us to the intricate details of what the application is actually supposed to look like in terms of UI and functionality. The reader is taken through creating a database and generating entities from these, as well as creating a login page. I felt that these two chapters were the weakest chapters of the book, seeing as they dwelt more on entities, JPA and entities generation and Seam’s identity and login management than it did on the particular component framework in question – Richfaces. This may be explained by the fact that Richfaces enables the user to hook into Hibernate Validator for DRY enforcement on the domain model, the user could have been spared the parts on how to create the database, etc as well as Seam’s login management features.

Subsequently, the next chapter gives some very good insight on skinning, and implementing a means for the user to dynamically change skins. It explains the Richfaces’s use of XCSS, a powerful xml based means of describing CSS properties, as well as how to use some of the default Plug-n-Skin examples.

Chapter 6 then takes us deep into a real ajax experience, showing us the lesser explored but very important features provided by the rich:dataTable and rich:column components, as well as the appropriate use of the datascroller component for displaying tabular data. It also talks extensively about other data iteration components like the DataGrid and the DataList. Additionally, it goes into detail on the appropriate use of ajax placeholders, especially on using Richfaces outputPanel in achieving really ajax based changes to UI after changes are made to content, instead of just using JSF’s rendered attribute only.

The next chapter then deals with using the RichEditor as well as how to implement Drag’n’Drop support in your application. Finally an in-depth score is done on using the Richfaces FileUpload component.

Chapter 8 and 9 deal into more detail with skinning support in Richfaces, including how to create your own Plug’n’Skin skins, which can be quite easy if you follow the author’s guidance.

Chapter 10 dwells on advanced ajax capabilities such as Poll, Push as well as the partial update of data iteration components in the UI.

Of course, this book would not have been complete without discussing the Richfaces Component Development Kit (CDK). From creating a maven project using Richfaces’s CDK archetypes, to creating your own user defined components and registering them with the application, this chapter covers it all.

It ends with an appendix giving a short description of all the components currently provided by the Richfaces team.

What I like about this book.

This book contains a lot of information about how to really take advantage of the ajax capabilities provided by the Richfaces framework, and makes important mention of very valuable points to look out for when ajaxifying your application. I highly commend the indepth knowledge of Richfaces placeholders and the use of attributes like ajaxSingle and process, which gives your JSF application a true ajax feel.

What I don’t like about this book.

I’m an avowed fan of JBoss Seam, but I’d have preferred the book didn’t delve so deeply into Seam to make it’s point. As i mentioned before, some chapters dealt more with Seam than with Richfaces, and those may actually be quite annoying to someone who is developing with say Spring Web Flow.

Packt Publishing has attempted to pull off a book that provides an indepth look at the 3.3 version of the popular JSF component library, Richfaces. In a lot of ways, I believe they have managed to do just that, give or take a few points.

For more information about the book, visit: http://www.packtpub.com/jboss-richfaces-3-3/book

Book Release : JBoss Tools 3 Developers Guide

JBoss Tools is an impressive set of open source Eclipse plugins that allow rapid development of enterprise and SOA based applications. It is developed by the JBoss group under the leadership of Max Rhydal Anderson.

It include support for rapid Hibernate, Seam, JSF and Richfaces, jBPM, JBoss ESB development and a lot more.There is no doubt that with such a compelling toolset, JBoss Tools stands out as the best set of tools for Ajax based JavaEE development that you can get for free and compares very well with those for pay.

To quickly point out how to harness the power of JBoss Tools for rapid application development, Packt Publishing has just announced their book JBoss Tools 3 Developers Guide by Anghel Leonard, scheduled to be released in April 2009. I believe this is the first book on JBoss Tools, and it has come at the right time for us developers.

Taking a hands on step by step approach to the 11 Chapters detailing each of the plugins at the time of writing, a developer is immediately delved into productive use of JBoss Tools in their project through a small project developed per chapter.

Of course, you will not expect to learn everything about Seam, Hibernate, Richfaces, JBoss-WS etc from the book simply because that is not the aim of it. However pointers are given to the right resources that will get you up to speed on them.

It has been fun being a technical reviewer of this book and I have to congratulate Leonard and the staff at Packt Publishing (Sarah and Leena) for their hardwork. I eagerly look forward to the release in April.

Don’t say I didn’t tell you how good it is, because this post is evidence that I have.

AJAX Magic With JSF DataTables in Seam 2.0

<Seam 2.0 on Tomcat … | Seam Portlet Bridge Released >
RichFaces 3.1.3.GA came with a some new controls, and the <rich:listShuttle/> was of some interest to me. However, for a particular use case I found it not sufficient for my needs, and had to roll out my own version of it with a little bit of Ajax to add.

Let me use this simple scenario to display what I needed to do. Assuming you were keeping a shopping cart. After users select an item they want to buy, you want them to specify the quantity of that item following which their total charge is calculated for them on the fly. This I thought of doing using two DataTables just like the <rich:listShuttle/> appears but on a sleeker (or is it cruder?) level.

I defined an interface with a simple set of methods which I felt would do the trick called InPlaceEditing.

public interface InPlaceEditing {

void editSelection();

void addSelection();

void removeSelection();

void cancelSelection();

}

Here is my”shopper” Seam component which implements the interface. Notice the use of 2 DataModels “products” and “selectedItems” and their corresponding DataModelSelections. This is to enable me select from one table to another.

@Name(“shopper”)

@Scope(ScopeType.CONVERSATION)

public class Shopper implements java.io.Serializable, InPlaceEditing {

private boolean edit;

@In

EntityManager entityManager;

@In

FacesMessages facesMessages;

 

@DataModel

List<Product> products;

 

@DataModelSelection(“products”)

private Product product;

 

@DataModel

private List<CartItem> selectedItems;

 

@DataModelSelection(“selectedItems”)

private CartItem selectedItem;

 

@Out(required = false)

private CartItem cartItem;

 

private BigDecimal total = new BigDecimal(0.0);

 

private static int count = 0;

 

@Begin(flushMode=FlushModeType.MANUAL)

public void beginShopping() {

products = entityManager.createQuery(“Select p from Product p”).getResultList();

selectedItems = new ArrayList<CartItem>();

}

The beginShopping() method starts a conversation and FlushMode is set to MANUAL. This means that all changes made to entities will be made persistent only if I call flush() on entityManager. Seam defaults to AUTO which means all changes to all managed entities are merged into the persistence context after every Seam action call. Trust me, for the purposes of this trick, you DON’T want automatic persistence context merging! Anyway this action populates the “products” DataModel with products already entered into the database. The result is the table below.

<a:outputPanel id=”productPanel”>

<rich:dataTable value=”#{products}” var=”product”>

<h:column>

<f:facet name=”header”>Product</f:facet>

<a:commandLink value=”#{product.name}” actionListener=”#{shopper.editSelection}” reRender=”editPanel”/>

</h:column>

<h:column>

<f:facet name=”header”>Price</f:facet>

#{product.price}

</h:column>

<h:column>

<f:facet name=”header”>Stock</f:facet>

#{product.stock}

</h:column>

</rich:dataTable>

</a:outputPanel>

Displayed editing panel

A look at the section of the shop.xhtml facelet shows an <a:commandLink/> – an Ajaxified version of the <h:commandButon/>. This supplies our “shopper” component with the selected product through the “product” data model selection through Ajax. Also take note of the “reRender” and <a:outputPanel/> tags, which specify which areas of our page should be Ajax refreshed after certain actions are called. In the editSelection() action a new CartItem (a join entity between a product and a shopping cart) containing the selected product is created and outjected.

public void editSelection() {

if (edit == false) {

cartItem = new CartItem();

cartItem.setId(count++);

cartItem.setProduct(product);

cartItem.setSelected(true);

return;

}

cartItem = selectedItem;

cartItem.setSelected(true);

edit = true;

}

I want to use the same action to represent selecting a new product (from “products”) and editing an already existing CartItem (from “selectedItems”). This is achieved with a private “edit” boolan field, which by default is false. Also since proper behaviour of DataModel and DataModelSelections depends on a properly implemented equals() method (i.e. instance uniqueness) I assigned artificial ids with the static “count” to each CartItem. Note that without setting flushMode to MANUAL, doing this will cause exceptions to be thrown down the lane somewhere. Finally setting the CartItem’s “selected” field to true enables the rendering of the panel in which the quantity of that product will be entered.

Displayed editing panel

 

DataTable with configured product

 

Once the quantity is specified, the addSelection() action is called. Because of the boolean “edit” we are able to determine if this is a new product selection or one from the existing list and respond appropriately, calculating the total cost of the users purchases so far and modifying the corresponding DataModels appropriately. In the case of an edit, the old computed cost of a product is deducted from the total, and re-computed and added to the total. To guarantee that the change in costs will reflect properly, the edited “cartItem” is removed and re-inserted in to the “selectedItems” DataModel.

It is also very easy to remove an already configured CartItem, again with the combination of <a:commandLink/> and the “selectedItem” DataModelSelection.

public void removeSelection() {

total.subtract(selectedItem.getCost());

selectedItems.remove(selectedItem);

}

Finally, the user may save their selection. This involves the creation of a new Cart object, which is persisted to get an ID and associated with the CartItems in the “selectedItems” DataModel. Note here that the fake Ids I generated for the CartItems will cause maximum trouble in the database, and so I set them to null to force the persistence context to generate proper Ids for them. Don’t forget the all important “entityManager.flush()” to make all changes permanent.

Completed Shopping

Finally I raise an event (“shopper.events.CartEdit”) which is being observed by the “shopList” Seam component’s list() action causing the “carts” DataModel to be refreshed with fresh data.

 

@Factory(“carts”)

@Observer(“shopper.events.CartEdited”)

public void list(){

Contexts.getSessionContext().set(“carts”, null);

carts = entityManager.createQuery(“Select c from Cart c”).getResultList();

}

Carts DataTable displaying products selected

The Job Is Done. This is use of Ajax and DataTables is quite simplistic, but I’ve used it in some really tight corners for some advance stuff. One scenario that I can see with this example is the requirement to remove a product from the list of products once selected and configured, or to put it back once it has been removed from the configured ones. Another will be how to edit an already persisted cart to remove or add products to that cart. I’ve just laid the foundation. With some tricks of your own, you should be able to achieve some serious magic.

JVM Language Storm

I’ve been following a bit the JVM languages that are struggling for pole position in the blogosphere of late. And the more time passes the more I’m convinced that the JVM is here to stay, even if folks might not be too happy about the Java language itself.

The stability of the JVM as well as the number of different technologies and APIs that it has evolved has made it a good breeding ground for languages that haven’t had the wide level of adoption as Java. Now with a greater push to support languages like Ruby (JRuby), Python (Jython) and totally new ones like Groovy and Scala, developers can still eat their cake and have it. There are even comments that JRuby is even faster than native Ruby, taking away the JVM startup time. And NetBeans is fast becoming the IDE for Ruby development. Support for other languages is being worked on aggressively in the IDE.

The focus on stability and platform independence is now reaping off for Sun Microsystems, and I’m happily exploring my options of getting on the bandwagon of new static and non-statically typed languages. The ability to still use all these language features as well as plugin to the technologies that Java and JVM already provide is very enticing.

Scala I must say looks very advanced and promising. A friend of mine said that sometimes exploring the language frightens him a bit because of the level of new, advanced and easy to use language features which are far from being just syntactic sugar. Scala is a statically typed language like Java, but it’s feature set is amazingly richer than Java’s. It is 100% compatible with Java. There’s a lot of noise on the blogosphere if Scala might not become the next flagship language for the JVM if Sun decides to move on from Java. Whichever they finally decide to choose, it will only be further to the advantage of the Java ecosystem.

The Scala support in IDEs is not as advanced as other languages like JRuby and Groovy. Here you’ll find a plugin for Groovy development in NetBeans. I’ve tried writing a few scripts with Groovy to get my feet wet and I’m beginning to enjoy it. This is basically because Groovy is so much like Java and the learning curve is not as steep. That is good for me, who hasn’t tried Ruby or Python before. My only issue is that IDE the support has not yet gotten to the point where I can productively use it in a project. However, I’m yet to explore the Eclipse support for it. Might be more advanced.

Proposals for JDK 7 include easy integration between Java and other JVM languages at JVM level. All the better I’d say. Fingers crossed, the JVM isn’t dieing anytime soon. It’s only getting better because unlike companies like Microsoft who just focus on doing things to please developers and gain market shares, Sun Microsystems takes it time to think throught the implications of it’s technology decisions, knowing that the future is what will vindicate it.