The following book should give you a much better overview and the details to create a linux systems for your custom board. This is a very intense topic: google "embedded linux", buy the book - read it, then try asking a more detailed question.
Building Embedded Linux Systems
FLTK is a very popular embedded toolkit. GTK with a minimum window handler is also fine. GNOME and KDE many times are too large for most custom boards - but if yours has the resources, it would make this project easier to complete; here GNOME would be the recommendation.