My tools of the trade

Using multiple processor cores in R-Projet

2012-01-16T19:07:00.004+01:00

My laptop is equipped with a Core i7 processor with 4 cores that can execute in parallel 8 processes. My R-Project computations use only one single core, if I use the default instructions. I have ended up by thinking that it is a pity that other cores are just sitting idle (sort of), instead of contributing to the speed of my computations, even if I do not run yet really heavy ones in my research. As a consequence, I have started to look for an easy way to use all cores in R-project. And, indeed, there is an easy solution to this problem. It uses the doMC library, and the instructions foreach and %dopar%.

For example, for computing linear models with different dependent variables and a given set of exogenous ones, one can do the following computations:

library(doMC) # There are other parallel computing libraries

registerDoMC() # You mud register one of them for foreach

getDoParWorkers() # Indicates you how many cores have been detected by registerDoMC()

Suppose that you have a dataset called mydata, containing the dependent variables y1, y2, y3, and the independent variables, x1,x2,x3.
We can execute in parallel the estimation of linear models of each y on the set of independent variables, by executing the following code:

myVariableList <- c("y1", "y2", "y3")

results <- foreach(i = 1:length(myVariableList),.errorhandling="stop",.inorder=TRUE)

%dopar% {

model <- lm(as.formula(paste(myVariableList[i],"~x1+x2+x3")),data=mydata)

return(model)

}

%dopar% executes these estimations on different cores, in parallel and a list of the estimated models is saved in the variable results.
We can now look at the characteristics of the estimated models, by printing them successively on the output of R:

for (i in 1:length(results)) { print(summary(results[[i]])) }

Voilà!

Of course, this possibility is especially useful for more complex computations, like stepwise regressions with many independent variables, that can take some time, or regression trees with big datasets, etc.

Broken XQuartz applications under OSX Lion

2012-01-07T14:21:00.000+01:00

So, my new laptop runs OSX Lion. I have been able to migrate my data from the old laptop (but the automatic process provided by the Migration assistant was a failure, Apple support had to help me to do it manually).

Many of the migrated applications have worked without a hitch as soon as the migration ended, but some other applications, like gretl -the GNU econometric software- and some wine-based windows tools, have refused to start. I was quite annoyed by this, but have not had any time to take care of it until today. I have installed wine using MacPorts (instead of the WineBottler that I normally use), the problem persisted. I have discovered PlayItOnMac (PION) implementation of Wine and installed it, but the installation stalled during the creation of the virtual disk needed by this soft.

Now, the support page of PION tells that you should start XQuartz to complete the installation process if you get stalled the first time. And I have tried to do launch it (it lives in Applications/Utilities), but it crashed by telling that the version I am trying to launch is maybe not compatible with my OS. Actually, that was an important clue. I have directly jumped on the XQuartz website and seen that there was a new version waiting for me. I have installed it and BINGO! All my Wine and X11 softs are working again.

So, if you have difficulties to launch Wine-based or XWindows softs after having upgraded to Lion (or on your new Mac under Lion), you should download and install the most recent version of XQuartz, from their web site.

Graphical interfaces (apps) for launching ipe7 and batch converting drawings from ipe6 to ipe7

2011-12-30T15:31:00.003+01:00

As promised below, I propose here some small graphical interfaces created with Platypus for:

Launching ipe. ipe7.app can be launched to open pdf/eps/ipe files created with ipe7. It contains the necessary environment variables (pointing to the standard TeX pdflatex path under TexLive - /usr/texbin/pdflatex) to allow the compilation of Latex text snippets. You can also associate .ipe files with this app and directly open them from finder. You can also right-click on an ipe7 pdf/eps file and « open with » this app.
Batch converting ipe6 files. ipe6to7-BatchConvert.app will open a small dropping windows on which you can … drop pdf/eps files created by ipe6. For each dropped file, it will save in the same folder as the dropped files two files with the same name as the dropped file (lets assume, filename.pdf), but different extensions: filename.xml is the extracted file;,you do not really need it and delete it if you desire; filename.ipe is the one you want. You can open it in ipe (using ipe7.app, for example ;-) ).
Converting and opening a single ipe6 file. ipe6to7-ConvertOpen.app will also open a dropping window where you will be able to drop a single ipe6 file (pdf or eps). The app will create the corresponding .xml and .ipe files in the same folder as the original file, and directly open the .ipe file in ipe7 for editing.

You can download these small apps from my web site. You will get a small zip archive that you can uncompress in any folder and copy in your Applications folder (but you do not need to, you can launch them from anywhere). I have tried to design them relatively generic, but ave not tried on another Mac. I have at least one small operation that is specific to my configuration, but it should not be blocking for you. Let me know if you meet any problems with these tools.

Note: These tools assume that you have installed on your Mac OSX:

ipe7 and ipe-tools using MacPorts (please see below my post on this installation process);
TexLive (a recent version, I use 2011).

Using Penn World Tables with R-Project, the easy way

2011-12-30T13:41:00.002+01:00

Penn World Tables (PWT) is a very nice data collection on economic growth. It covers a large set of countries (from their web site):

« The Penn World Table provides purchasing power parity and national income accounts converted to international prices for 189 countries/territories for some or all of the years 1950-2009. The European Union or the OECD provide more detailed purchasing power and real product estimates for their countries and the World Bank makes current price estimates for most PWT countries at the GDP level. »

I use these data for creating graphics for my economic growth course. My workflow was based on importing them in csv format into R-Project. But I have very recently discovered that there is much better way of using them ;-) Just loading the pwt library in your R-project code, thanks to Achim Zeileis, Guan Yang who provide this library. You must first install it from CRAN, using the usual R command for this. Once it is installed, it is enough to run the following commands to gain access to the data contained in PWT 7:

library(pwt)

data(pwt7.0)

You can check the names of the included variables:

names(pwt7.0)

And the help of the package gives you the exact definition of these variables:

help("pwt7.0")

If you prefer, you can use a more user-friendly name for this table:

myData <- pwt7.0

And clean the row names:

row.names(total) <- NULL

And voilà!

Simple and easy, thanks to Achim and Guan :-)

Installing ipe 7 under OSX Lion

2011-12-30T12:22:00.002+01:00

After having tried other solutions for my pdflatex graphics, I am reconsidering again ipe (see below my post about this very nice and useful program that allows you to put latex instructions in your drawings). My initial wanderings were motivated by the unavailability of a reliable ipe version under Snow Leopard. I have been able to compile a 6pre28 version (see below), but it was nor as reliable as I desired and it became completely unusable as soon as I have switched to Lion (mainly because of the Qt framework version I have used to compile it). I have also played with versions available in MacPorts and this is the solution I will propose to you for installing ipe7 under Lion.

A/ Installing MacPorts. First of all, you must download and install MacPorts. It could look daunting at a first look, but the process is quite painless (if you already have MacPorts on your Mac, go to the step B below):

You first download the dmg archive corresponding to your system (Lion for me) from the install page of Macports;
Double-click on the downloaded MacPorts-2.0.3-10.7-Lion.dmg file to mount it in Finder and open it in the finder (I assume that you know how to use a dmg file);
Switch in Finder to the folder corresponding to the dmg volume and double-click on c to launch the installer. You follow the instructions (they are very straightforward). MacPorts installs itself, by default, in /opt/local folder on your system hard disk.
Once the installation finishes, restart your Mac to be sure that everything (PATH and all will be correctly fixed).
When your system is back, open a Terminal session and type: « sudo port selfupdate » to update the repository of your macport installation with the list of the most recent packages (you will have to type your administrative password for this).

B/ Installing ipe and and ipe-tools. You are now ready to install ipe :

You can check the availability of ipe, by typing « port search ipe »;

You will get the list of all packages that have « ipe » as a part of their name. The two packages that interest us are: ipe @7.0.14 (graphics) (The Ipe extensible drawing editor, the main program) and ipe-tools @20110916 (graphics) (Tools for the Ipe extensible drawing editor, utility tools for converting old ipe drawings to ipe7 - see below about this point).

You install them by typing in the Terminal:
- sudo port install ipe
- sudo port install ipe-tools

The installation process will download other all necessary MacPorts packages, and it will take some time depending on the sped of your internet connection and of your Mac. If ever the process stop, you just relaunch the installation instruction. MacPorts will in general give you tips for solving the problem, if you meet any (generally it just works, you must be patient, that’s all).

You will have a bunch of command in the /opt/local/bin folder now, and the ones which interest us are:
- ipe (the main program, ipe7);
- ipeextract (the utility for extracting ipe instruction from ipe created PDF or EPS files -> ipeextract myfile.pdf gives myfile.xml as output);
- ipe6upgrade (the utility for upgrading ipe6 drawings to ipe7 format -> ipe6upgrade myfile.xml gives myfile.ipe as output);
- These programs are supposed to be launched in the Terminal, but I have created solution for launching them from the Applications menu (using Platypus - see below; I will do another post on this point).
- MacPorts will also install a MacPorts menu in your Applications menu. There will even be an « ipe » item in this menu and it will launch the program. You will be able to draw with it, but, since it does not correctly import the environment variables, you will not be able to run Latex from it. Everything works better if you launch ipe from the command line (or if you use the utilities I have developed using Platypus).

Now you have a working version of ipe7 on your Mac. Have fun!

Switching to GooglePlus

2011-08-15T15:59:00.003+02:00

Posterous does not accept autoposts by mail from/to GooglePlus, so some of my articles can now only be seen on my G+ page.

Moreover, given the complexity of managing Facebook, Twitter, a blog (Posterous was a potential solution for merging these streams) and now G+, I will progressively group everything in the latter stream.

You can read my Google+ stream from this address: My GooglePlus.

My Posterous: http://yildizoglu.posterous.com

A very nice free tool for OSX: Platypus helps you to make your scripts more user-friendly

2011-02-24T19:41:00.001+01:00

I have just discovered a very nice tool. I was looking for a solution that would allow me to include software installed via MacPorts in my Applications menu. I was looking for more than a simple symbolic link to a bash script. Something nicer, with an icon and some interactivity. Then I have found a reference to Platypus on a forum.

Platypus allows you to pack any command-line based software into an .app bundle with a nice icon, or a droplet on which you can "drop" a set of files for batch processing.
You can bundle many different programs as an App: your scripts - Perl, Php, Python, AppleScript- , command line tools included in OSX or installed via MacPorts,etc.

Platypus is quite simple to use and comes with nice and helpful documentation.
If you like it, you can support the project with a donation.

My Posterous: http://yildizoglu.posterous.com

A nice tool for presenting dynamic data for teaching: Gapminder

2011-01-23T20:19:00.004+01:00

This time I will write on my first impression about a gorgeous teaching tool that I have discovered thanks to a TED video by Hans Rosling: Gapminder.
Gapminder is a web site that allows dynamic presentation of many interesting time series (economic indicators, demographics, education, technology). It is also possible to download an off-line version made with Adobe Air.
It is very straightforward to use (just click to select the data to show on both axes, for the size of the bubbles and the categorization (geographic regions, countries, etc.) for the coloring of the points. You can animate the evolution of the variables in time and show very lively graphs to your students.
In my courses on economic growth, I generally confront theories with empirics and this tool will give me much better speaking graphs.

Unfortunately, there is no way to include your data in the tool and the real utility of it will finally depend on the data sets you need. But GapMinder web site is kind enough to show how to animate your data using Google Motion Chart. I was not aware of the existence of this tool...

LyX 2.0.0 beta 2 released

2010-12-14T19:18:00.001+01:00

LaTeX Community News
10 décembre 2010 16:24
by mailrobot@latex-community.org (Stefan_K)

LyX 2.0.0 beta 2 released

The second beta version of LyX 2.0.0 has been released.

News provided by

The second beta version of LyX 2.0.0 has been released. Testers of the first beta are encouraged to test the new version.

The current stable release for any serious work remains 1.6.

For a list of new features have a look at a previous blog post or visit

the List of new features and changes,
the LyX Homepage,
the announcement of the 2nd beta on the mailing list,
the 2nd beta on the FTP server.

This text is available in German. Dieser Text ist auch in Deutsch verfügbar.

LaTeX

LaTeX Community News 10 décembre 2010 16:24 by mailrobot@latex-community.org (Stefan_K) LyX 2.0.0 beta 2 released The second beta version of LyX 2.0.0 has been released. News provided by The second beta version of LyX 2.0.0 has been released. Testers of the first beta ar ... http://yildizoglu.posterous.com/lyx-200-beta-2-released My Posterous: http://yildizoglu.posterous.com {{hash_tags}}

Moving time: Adopting Posterous for blogging

2010-12-10T22:29:00.002+01:00

Hello,

I am progressively moving this blog to Posterous: http://yildizoglu.posterous.com. I will keep its focus on the tools I use in mys research projects, but I will also post, from time to time, articles on other subjects and photographic experiments.

I hope that you will continue to appreciate it. Posterous provides interesting new possibilities for sharing articles on other social networks. Tumblr was also another possibility but I Posterous better articulates with my way of blogging.

Some of my new articles on Posterous will also be automatically posted on this blog (when they are in-line with its focus), but they will not be necessarily nicely typeset (because of automatic posting). So, ideally, I would invite you to follow the Posterous blog.

Compiling Ipe 6.0pre32 on Mac OS X 10.6 (Snow

2010-08-10T21:07:00.000+02:00

Using these instructions proposed by Sherif Ghali on the ipe mailing list, I have been able to compile it for OSX 1.6.4. Thanks a lot Sherif!

You can read Sherif’s instructions on the IPE mailing list archives.

Installing Dictionnaire de de l'Académie Française in the OSX Dictionary application

2010-08-10T20:57:00.000+02:00

You can download from this link a zip file that contains a packager for this dictionary. Thanks a lot to Etresoft! Unzip the archive and double-click on the « Dictionnaire-AcademieFrancaise_1935.pkg » file. After installation, open the Dictionary application and activate this dictionary in the preferences. Voilà!

Thanks to MacOSXHints for this tip!

Replacing PowerPro for Windows with OSX tools

2010-07-31T02:14:00.000+02:00

As I have written in my initial switching post, one of the softs that i am missing under OSX is the magical PowerPro (PP) program for Windows. PowerPro is a very small program but it is a real Swiss Army Knife with a very rich set of functionalities (I am sure that during all years I have used it, I have not been able to discover all of them!). I have first find Butler to replace PowerPro for some of the facilities, but inserting text templates (like different greetings in mails or small LateX snippets) was rather painful. First open the shortcuts screen of Butler, then type the alias for the snippet, then type return, etc. Under PowerPro, I could just type ;prop and PP would replace it with

\begin{proposition}

\end{proposition}

I was used to have at mys disposition a whole set of such snippets for different softs, and was missing them under OSX.

Enter KeyBoard Maestro (KM)! It is definitely not as powerful as PowerPro, but it brings the exact function that I was missing. I can now type ;prop and get the corresponding snippet. Or type CTRL+F and activate the French spelling dictionary in FireFox (combining the very nice Quick Locale Switcher plugin with Keyboard Maestro). KM can do many other tricks (launching programs, resizing their windows, opening some default document by simulating clicks on menu items, etc.). It is not free, but it is not unaffordable either.

You can check its features from this page and, if you are convinced by its possibilities, you can order it from this page.

Living with the reflective screen of the MacBook Pro

2010-07-31T01:46:00.000+02:00

My switch to Mac is now complete and I very rarely start Parallels Windows 7 VM. One of the major difficulties I have met with the MBPro is the eye (and brain) strain caused by the very reflective screen of this laptop, when I work outside, under high sun light. Apple indeed does not propose a matte option for the MBPro 13’. Since I have used anti-reflective filters for my PDAs, I knew that such a solution exists. After having done some research on the Net, I have ordered a Nushield DayVue Screen Protector that I have received today.

Installing the filter has been quite a hassle, as its is always the case with this kind of protections. The important point is to install it without capturing any dust between the filter and the screen. I have spent some time to clean the screen of the laptop. I had to admit that I would not be able to eliminate all traces from the glass, since at least one of them has completely resisted all my rubbing efforts. Since it was a very small bump point, I have decide to install the filter anyway. You must be very careful not to capture supplementary dust during the installation. Consequently, you need a dust free environment. Doing the installation in the bathroom, after a hot shower is the trick, since the humidity captures dust particles. With the help of my daughter (installing a 13’ filter is a little bit more complex than a 3’ filter I have used for my PDAs!), we have been able to make quite a nice installation, without many particles and bubbles. When I have finished pushing all bubbles to the borders of the filter, using a credit card, I was ready to test my new screen in the garden:

Anti-reflection works really! I continue to see some reflective light on the screen, but it is much less disturbing. The result was quite impressive indeed.
The filter does not reduce much the light emitted by the screen. I have read for other filters that their users were complaining about the necessity of using the screen with maximal brightness. I have not observed such a negative effect.
The filter does not introduce any distortion in the screen. In fact, I am totally unable to see it on the screen (except in on the very small dust point that I have not been able to eliminate from my screen).
My experience with this filter is very satisfactory until now. If I observe new problems, I will edit this post to share them with you.

Switching to Mac OSX (Snow Leopard)

2010-04-27T22:08:00.003+02:00

I am starting to migrate to Mac OSX my working environment. I am an old Linux user for my servers but I have never been completely convinced by the Desktops for Linux (mainly KDE and Gnome) and gone back to Windows with the introduction of its first usable version (XP). I have deliberately missed the Vista wagon, and have not been impressed by Windows 7. Being tired by the quirks of XP, I wanted to adopt a more reliable and modern interface. OSX seems to me to comply with this demand, and the Unix that is behind the interface is quite a standard one, quite easy to understand for a person used to Linux. I am a happy camper concerning the very well thought interface of OSX.

Migrating my working environment to OSX has not been completely simple. Before the switch, I have, of course, checked that this migration is broadly possible. Here I indicate what was very simple, what was more bumpy, and the points for which I have not yet found a completely satisfactory solution.

Immediate transition for:

Internet tools, starting with Firefox (I really need it for Zotero and other tools that do not exist under Safari or Chrome);
Latex compilation, just install TexLive, it is perfect.
For many aspects of Tex on OSX, check the wiki
Simple Latex editors: TexMaker is a very nice multi-platform editor. Aquamacs (Xemacs with an Aqua interface) comes standard with AUCTex, which a very powerful Tex editing and compiling tool.
Graphical Latex editors: I was using Lyx under Windows, there is an OSX version.
Mendeley Desktop (very nice, bibtex compatible library manager) has an OSX version.
Statistical tools: R-project offers an OSX version, perfect for me; it was my main tool under Windows. I am using Aquamacs also for R, since it comes with ESS (Emacs speaks statistics), which is also very powerful. Gretl (econometrics) works under OSX.
Java programming: Eclipse works under OSX.
NetLogo works under OSX.
Bitmap graphics: Gimp also offers an OSX version.
Vectorial graphics: JPicEDT is written in Java, so you can use it also under OSX, without any problem
MS Office: I am using Office 2008.
Picasa has an OSX version (I prefer it to iPhoto, probably because I am too lazy to replace an old software that satisfies me).
Skype has on OSX version.
Evernote works nicely with OSX.
Freemind mind map software also works under OSX (Java).
Zipeg is a nice and free zip archive manager.

Problems begin:

One difficult point has been to find a nice substitute to PowerPro (this very nifty and talented swiss-army knife utility for managing many aspects of windows and automatizing many tasks). Butler is able to replace some of them. I am learning AppleScript for some other aspects. Don't call this an easy transition ;-)
Lyx has problems to import some big Latex files I have written with Scientific Word, so I continue to use the latter under a Parallel's virtual machine :-( I would really have an OSX version of SWP, even if I love Lyx...
Unfortunately Entourage in Office 2008 is not compatible with Outlook (such a stupid strategy!); I am waiting Office 2010 that will reintroduce Outlook. I like quite well Apple-Mail but my several years old mail archive is under Outlook format and I do not want to loose mails during the conversion process...
Making Inkscape + TexText work under OSX has been a pain. I do not even remember how I have been able to do it (I have followed and mixed several forum post on this topic).
Ipe graphics editor for Latex does not propose an OSX version out of the box. It seems possible to compile it for OSX, but I have not had yet the time to do it (it seems a little bit painful).
Xara X drawing software for vector graphics. I am really missing this one. I had to install it in the virtual machine.

Nice surprises:

Bibliography management for bibtex: I like Jabref (it is multi-platform), but Bibdesk is a very nice reference management tool.
Backup: Time machine is very nice and you have also nice imaging tools under OSX. You can use TimeMachineEditor to customize some of its aspects. Backblaze also works for OSX and I have been able to transfer my subscription to the new machine without any problem (be just ready to loose all your old Windows backups).
The Agenda of OSX is very nice and simply synchronizes with Google calendar.
Bean is a very nice and light Word file editor.
Skim is a very useful free Acrobat viewer (and editor). Preview in OSX also very useful (deleting pages, etc.). FormulatePro allows the filling of Acrobat forms.
MyMind is a very nice idea editor.
WineBottler (free) is a very nice Wine version that allows the installation of some small Windows programs under OSX. I have tested CrossOverX, but have not really been convinced by its advantages.
OSX comes with many small and nice utilities (managing hard disk, writing DVDs, etc.).

Conclusion: I am using my virtual machine only for Scientific Word (only for some files, all new files are written under LyX or Aquamacs), Ipe and Xara X. I am quite happy with the coherent working environment provided by OSX. I am not ready to switch back to Windows for some time... :-)

Sumatra: Quick PDF viewer for Windows

2009-09-10T23:28:00.003+02:00

If you use pdfLaTeX under Windows, you are maybe tired of waiting Acrobat reader to open, each time you compile your document and want to check the preview. I like Acrobat for its professional features but I rarely need them. For all other occasions where I just want to quickly check a PDF file, Acrobat is just a pain to launch.
I have tested FoxReader in the past but I was not fullt happy with it. I think that I have now found the perfect quick PDF reader, thanks to a post in the Lyx's users-forum: Sumatra PDF Reader. It has even a full screen mode that works quite well (even Acrobat's better fills the screen). It gets along quite well with any LaTeX editor (Lyx, TexMaker, Scientific Word, etc.).
It is also quite easy to drive using keyboard shortcuts, as you can learn this in the "manual".

A very nice and free text editor: Notepad++

2009-06-19T00:36:00.004+02:00

Notepad++ is a free text editor with a lot of very useful functionality and it has replaced PSPAD for me.

You can edit source code in many languages, benefitting from syntax highlighting and auto-insertion of the main commands.

It places itself in the context menu of the explorer, for facilitating the quick opening of any text files from a right click in the explorer. It is difficult to exhaustively indicate all its functions.

It is based on the Scintilla platform; it is both very easy to use and very powerful.

Here are the features of Notepad++, as indicated on the web site:

Syntax Highlighting and Syntax Folding
WYSIWYG
User Defined Syntax Highlighting
Auto-completion
Multi-Document
Multi-View
Regular Expression Search/Replace supported
Full Drag 'N' Drop supported
Dynamic position of Views
File Status Auto-detection
Zoom in and zoom out
Multi-Language environment supported
Bookmark
Brace and Indent guideline Highlighting
Macro recording and playback

Link: http://notepad-plus.sourceforge.net/uk/site.htm

ChkTex with LyX and Miktex 2.7 under Windows XP

2009-02-25T22:03:00.005+01:00

ChkTeX is a LaTeX utility for... checking your Latex syntax. It is maintained by Baruch Even.
LyX can use ChkTex utility. But Miktex does not come with this utility.

I propose here a way to activate this functionality using the chktex binary that is distributed in the TeXLive LaTeX system. This is quite simple indeed.

I assume that you have installed Miktex in the "C:\MiKTeX2.7" folder.

First download the chkTexFiles.zip archive from my web server. It contains the necessary files for the installation of chktex under Miktex.
This archive contains the following files:

.chktexrc
ChkTeX.exe
tl90geturl.dll
tl90gnu.dll
tl90kpse.dll

The first file (.chktexrc) is the chktex configuration file. You must put it under in the following folder: "C:\MiKTeX2.7\tex\chktex"
The other four files are the executable for chktex and the necessary libraries. You must put them in the folder C:\MiKTeX2.7\miktex\bin.

You must now tell LyX how to find these files.

You activate "Tools/Preferences" menu item in LyX, and in the Output/Latex item in the left menu, you must give the following command for ChkTex:
ChkTeX.exe -l "C:\MiKTeX 2.7\tex\chktex\.chktexrc" -n1 -n3 -n6 -n9 -n22 -n25 -n30 -n38
I must admit that I have not dug all the options for ChkTeX, you can probably finetune them using the ChkTeX documentation but these default options work on my installation. If you execute chktex -h in a Dos box, you will get an output that will indicate the role of different options.

Once you save the configuration (do not forget to click on the Save button in the preferences box ;-) ), you can execute "Tools/Check Tex" menu item and it will indicate the potential problems with your LaTeX file. Don't be afraid by the results! :-)

ChkTeX source files are also available from CTAN.

Lyx and Sweave under Windows XP

2009-02-15T23:29:00.011+01:00

These instructions propose a solution for making Sweave work under Windows XP, using Lyx 1.6.1 and R-project 2.8.x.

Adapted from the instructions provided by Paul Johnson and Cheng-shan (Frank) Liu:
(see http://n2.nabble.com/Converter-failure-with-Sweave-td479669.html)

These instructions correct some small problems that were impeding the original instructions from working under the recent version of R that I use. I also take into account the fact that the default installation folders of R and Lyx are under Program Files, a path that contains a space, and can cause problems.

We will suppose that Lyx 1.6.1 is installed in C:\Program Files\LyX16 and R in C:\Program Files\R\R-2.8.1 (their default folders under Windows).

Place noweb.sty and sweave.sty (part of the R installation - see the share\texmf subfolder of R, see the next instruction) in a folder that can be find by your Latex installation (under texmf-local for example).
Copy the content of the C:\Program Files\R\R-2.8.1\share\texmf folder in the previous folder or in another folder under your texmf tree.
Refresh the file catalogue of Latex (execute mktexlsr for TexLive in a Dos command box, for example). You can now check the placement of these files by executing kpsewhich noweb.sty in a Dos command box.
Reconfigure Lyx (go to Edit-> Reconfigure). Check if you have document class "article(noweb)" or "article(Sweave noweb)" (in Document-> Settings->Document class). If not, you will need to reinstall Lyx.
Create a batch script called Rweave.bat and put it in the folder C:\Program Files\Lyx16\bin\. You can create this file using notepad or any other text editor (PSPad is a very nice and free one). The file should contain a one line instruction:
"C:\Program Files\R\R-2.8.1\bin\Rterm" --no-save --args "%1" < "C:/Program Files/LyX16/bin/MakeSweave.R" > "%1.log"
Create a R file MakeSweave.R with the following lines and put it in C:\Program Files\Lyx16\bin\:

library(tools)

args <- commandArgs() filename <- args[length(args)] Sweave(filename) basename <- sub("\\.(Rnw|Rtex|nw)$", "",filename) texi2dvi(paste(basename, ".tex",sep=""), pdf=TRUE)
You must now configure in Lyx the converter for noweb files. Go to Edit->Preferences->File Handlers->Converters. In the "From" pulldown, choose Noweb. In the "To" pulldown, choose PDF (pdflatex).In the box called "Converter" type "Rweave $$i" without the quotation marks. If necessary, click the “Modify” button and save the new command.
You can now test your installation by opening an example file that contains R scraps. For example Paul Johnson's Gamma distribution lyx document.
You can typeset this document using the pdf icon and the resulting file should open in the acrobat viewer you have configured in Lyx.
You should be able to read the results of computations and see the plots.

Links:

Sweave web page
Lyx Wiki page for Sweave
Using LyX with Sweave by Gregor Gorjanc
Complementary instructions by Jeff Laake (if mines are not enough in your case)

A free and powerful text processor : LyX

2009-02-14T20:46:00.005+01:00

LyX is a nearly WSYWIG front end for Latex. It constitutes a very credible open source alternative to my other preferred (commercial and quite expensive) Latex editor (Scientific Word).

I have been following LyX's development for some years, without being fully convinced by its facilities. I consider that it now constitutes a very useful front-end to Latex: it is very stable and it fully interfaces with all the facilities characterizing Latex (typesetting mathematics, bibliography management, cross references, graphics, integrating results of statistical computations using R-Project - thanks to the Sweave interface, etc). It definitely offers today more functionality than Scientific Word (see below).
As a consequence, I do not have any reserve in advising you to test LyX, I think you will love it. Binary versions are available for all plateforms (Windows, Linux, Mac OS X).

Web site
Wiki

A very nice file explorer for Windows: Q-dir

2008-09-08T21:32:00.003+02:00

This time I am presenting a tool that is not directly related to the production of documents with LaTeX. Q-dir a very nice and free file manager that I have discovered recently. It manages to be very small and, nevertheless, very flexible and efficient. For me, the main two very nice properties are the possibility of

opening several folders side by side for intensive files copying;

saving typical folder configurations as favourites and open them all in the same time by calling the corresponding favourite;

I have for example a fovourite for all folders corresponding to LateX file editing and another for HTML editing. Switching from one to the other is very quick and easy. You can also create favourites to your important folder for opening them directly, by two clicks of the mouse (one for calling the function and another for chosing the favourite folder.

It has of course many other very nice features. You cab also install it on a USB stick.
I invite you to check the website and download it for learning more about the possibilities it offers.
To repeat the slogan of it's creator: "Once Q-Dir, always Q-Dir ;)!"

Editing Latex Files under Windows with LatexEditor

2008-08-18T01:29:00.003+02:00

Another very interesting tool for editing and compiling Latex files under Windows is LatexEditor (LED). It has the advantage over Scientific Word of being free.

It is not WYSIWYG or WYSIWYM (WhatYouSeeIsWhatYouMean) but it included a DVI viewer that can show the compiled document just next to the Latex source window.
LED also has many customizable tool bars (and you should customize them because the default configuration is not really as useful as what you get when you configure them to your needs) that lighten the burden of text typing .
It also integrates quite powerful spell check dictionaries and thesaurus. It can also convert files between different code pages during the opening and saving of the files: you can have an 8-bits version with accents on the screen (allowing easy spell check and readibility) and keep on the hard disk the 7-bits version where é is coded, as usual in Latex, \'{e}. You must customize filters to obtain such functionality (see, for example, the filter for French characters I have proposed on the LED support forums).

Migrating to a new laptop

2008-08-18T00:43:00.000+02:00

I have recently received a new laptop and I had to proceed with the migration of my stupidly complex XP Pro installation to the new computer. Such a migration normally takes to me at least one whole week if I proceed through an install from the scratch. Given that I am terribly busy these days, I was desperately looking for a solution to make easier this migration. For years I have effectively been asking myself why someone (why not Microsoft? It's basic data transfer tool is jut ... too basic!) cannot invent a convenient tool for transferring the complete software setup from one computer to another.
This time I was motivated to better search and I have discovered that such a miraculous tool now exists and it is called Move Me by Spearit Software.

You install Move Me on the original computer and on the new computer, you plug both of them on the same network (it also handles transfer through USB or parallel connections) and start it on both of them. Then a wizard allows you to easily setup the system. It compares files on both systems and decides to transfer necessary new files to the new computer.
This part of the process could seem just stupid to you and you could think that any backup of copy software could do it. If you effectively try this by yourself, you will very quickly discover that it is not enough to transfer program files; many libraries and other configuration files will be missing in the Windows folder of the new computer and the programs will not in general work correctly. If you try to transfer also the files in the Windows folder, then you will very quickly see that this is not a good idea since it will mess up the Windows installation of the new computer. Another source of problem will be missing Registry entries and even if you are able to intelligently transfer just the necessary DLL files, your programs will not work or you will have to reconfigure them to your taste from scratch. This just normally takes a huge amount of time too.
This is where Move Me shines: it intelligently transfer system files installed by the programs, without messing with the Windows installation and it transfers the registry settings and imports them in the Registry of the new computer.
When the transfer finishes (that can take quite some time in a complex installation like mine), you boot the new compute and find your original configuration with the usual softwares and your customization. This is just magical and marvellous!

Caveats :

Move Me is not free. You pay a fix fee for each transfer (I have paid $47 including VAT). When you pay this amount, you do not buy the software but just the right to migrate from one single particular computer (but you can migrate to as many computers as you want from this computer - so the licence you but sticks to the original laptop). I have thought that spending 50 bucks for economizing one week's job was reasonable and I do not regret it: In one night my new laptop became fully functional!).
Move me does not transfer the active services and hardware configurations in order to preserve the system of the destination laptop. This means that you will have to reinstall some software in the new computer, mainly the softs that are intricately articulated with the operating system (eg. Anti-virus, firewall, virtual printers, Adobe Acrobat, Diskkeeper, etc.). It is even better to install first these softs on the new computer before beginning the migration.
Move Me does not duplicate user accounts existing in the original computer onto the new laptop, but each user can log in the original computer to migrate his/her personal data. I have found that the migration was really successful if you log under exactly the same administrative account (with the same name, and hence the same Documents and Settings folder name). This makes the migration much more robust and reliable.

I know now that I can migrate to a new laptop without spending 7-10 days for restoring my working environment on it. This is just marvelous and I thank Spearit for its inventiveness.
Note: The same software seems to also be commercialized by Laplink.

Update (August 18, 2008): MoveMe does not seem to handle the migration towards Windows Vista, while the Laplink version (PcMover) pretends to do. I have not yet tested this possibility.

Writing articles in English

2006-05-07T10:55:00.001+02:00

If you write in English (this is probably the case if you are reading this page :-) ), you can definitely learn something from The Elements of Style. This small text by William Strunk, Jr. corrects many potential errors we can make when we write in English, especially if we are not native speakers (even if the book is not originally dedicated to foreign writers). But some adverse opinion also exists about this book.

Another very interesting small book about style is Simple & Direct by Jacques Barzun (published by Quill). Again, this book incites you to clarify your ideas in order to craft sentences with a clear meaning.

More on the grammar side, I would also recommend Practical English Usage by Michael Swan (Oxford University Press). It is really very easy to use thanks to a very intelligent organization. A small excerpt from the table of contents:
173 - each
174- each and every
175 - each other and one another
...

If you mainly write scientific articles, you could benefit from Successful Scientific Writing by Matthews, Bowen and Mathews (Cambridge University Press). This book is dedicated to biological and medical sciences but the tips it contains can help for articles in any discipline (I am an economist and I have learned a lot of tricks from it, even if my writing remains quite awful in English :-)).

And if your mother tongue is French, just a single major tip: forget the passive voice, it terribly blurs the significance of sentences in English. Passive voice also gives rise to terribly cumbersome sentences very prone to errors...

And, if English is your third language (as in my case), I wish you good luck ;-)

Other interesting pages:

Open source statistical analysis plateform: R-project

2006-04-09T13:00:00.001+02:00

R is the statistical open source software that I use for analyzing the results of my economic simulations and other statistical data. R is a very powerful tool developed by a considerable and very active community. It is not really very user-friendly when you use it out of the box, through the command line, but some of the included packages try to make easier your life (like the RCommander package that is visible on the screenshot above). Other visual user interface projects are also in development. For example, Sciviews already proposes two nice tools: a very flexible and powerful script editor Tinn--R and a graphical gui that replaces the standard Rgui, Sciviews-R. The latter enriches the standard Rgui with some widgets that give an easy access to obkects in memory, for example, and a more flexible editor (but it is also possible to use Tinn-R in combination with this GUI).
The main website of R is the www.r-project.org. You can also download R, other packages and tools from a CRAN repository close to you (check the list of the mirror sites here).

From the website of R:

R is a language and environment for statistical computing and graphics. It is a GNU project which is similar to the S language and environment which was developed at Bell Laboratories (formerly AT&T, now Lucent Technologies) by John Chambers and colleagues. R can be considered as a different implementation of S. There are some important differences, but much code written for S runs unaltered under R.

R provides a wide variety of statistical (linear and nonlinear modelling, classical statistical tests, time-series analysis, classification, clustering, ...) and graphical techniques, and is highly extensible. The S language is often the vehicle of choice for research in statistical methodology, and R provides an Open Source route to participation in that activity.

One of R's strengths is the ease with which well-designed publication-quality plots can be produced, including mathematical symbols and formulae where needed. Great care has been taken over the defaults for the minor design choices in graphics, but the user retains full control.

I really invite you to download R and begin to play with it. You can find very useful tutorials on R on the website, in the Contributed documentation page. Check, for example, “Using R for Data Analysis and Graphics - Introduction, Examples and Commentary” by John Maindonald. You can download the PDF file from this link.

Editing LaTeX files under Windows: Scientific WorkPlace

2006-04-06T17:40:00.002+02:00

This one is not a freeware or open source (it is even quite expensive - especially in France) :-( but I find that SWP is the most powerful, compatible and user-friendly LaTeX editor that we actually have. There are other open source competitors like the very beautiful TeXMacs or Lyx , but he first is not as compatible as SWP and the second is not yet as flexible as SWP, especially concerning the editing of mathematical expressions. Another nice open source but not at all WYSWYG editor is TeXnicCenter.

SWP can save your files in a compatible format that is called "portable latex" and you can easily compile your files using a standard LaTeX motor (I, for example, use Miktex instead of the specific compiler that comes with SWP: TrueTex).

Mackichan provides two versions of this editor: Scientific Word (the editor only) and Scientific Workplace. The workplace version is more expensive because it comes with a algebraic computation system: Mupad. If you have an official version of Maple, you can also use the kernel provided by the latter (I find it more powerful by the way). In fact I do not possess a complete copy of Maple but I have conserved the last kernel that I have had with an old version of SWP (SWP was used to come with Maple) and I configure SWP for this kernel and this sea tup perfectly works. As consequence, I can do computations or plot expressions directly from SWP.

Drawing figures for LaTeX - jpicEdt - Java Picture editor

2006-04-02T18:43:00.000+02:00

jPicEdt is a very user-friendly open source software for creating and editing your drawings for LaTeX documents. Its main strength is, in my opinion, its capacity to generate drawings based on the very nice package PSTRICKS. This is gives a very powerful environment for creating sophisticated drawings.

From the web site (www.jpicedt.org):

jPicEdt can generate LaTeX, eepic and PsTricks code from a user-friendly graphical interface! This may include:

* Every graphical element allowed by the picture environment commands: lines, arrows, circles, boxes.
* Emulated elements, for example lines of any slope (which LaTeX doesn't support natively), circles of any size, ellipses, arcs and polygons. This makes a strong use of the \multiput command.
* Nearly every object allowed by the epic/eepic packages (excluding grids and textured filling): dashed lines, filled (whitened, blackened or shaded) ellipses or polygons, arcs,... Using this package is optional.
* As of version 1.3.2, nearly every object allowed by the pstricks.sty package (this include : filling with colours, hatches, textures, setting various stroke parameters...). Support for pstnode.sty and other related packages is underway.

jPicEdt can parse LaTeX files that include a \begin{picture}...\end{picture} (or \begin{pspicture}...\end{pspicture}) block, and is able to interpret a great deal of commands, either LaTeX-, PsTricks- or eepic-compliant (this allows in particular to load files generated by GnuPlot with a gset term eepic command).
jPicEdt is written in JAVA. As a result, it can run on any platform where a JVM (Java Virtual Machine) is installed (nearly every Unix flavors, plus Window 9x/2000/XP and MacOS, at least...)

If you configure jpicEdt for compiling the drawings with LaTeX and dvips, you can very easily convert them to PDF for also using them with PDFLaTeX.

Drawing figures for Latex - Ipe extensible drawing editor

2006-03-31T15:19:00.000+02:00

Ipe extensible drawing editor is a very nice tool for drawing figures for LaTeX. It is developed by Otfried Cheong. The source code and the binaries for Windows and Linux can be downloaded from ipe.compgeom.org.

The features of ipe are (from the web site):

* Entry of text as LaTeX source code. This makes it easy to enter mathematical expressions, and to reuse the LaTeX-macros of the main document. In the display text is displayed as it will appear in the figure.
* Produces pure Postscript/PDF, including the text. Ipe converts the LaTeX-source to PDF or Postscript when the file is saved.
* It is easy to align objects with respect to each other (for instance, to place a point on the intersection of two lines, or to draw a circle through three given points) using various snapping modes.
* Users can provide Ipelets (Ipe plug-ins) to add functionality to Ipe. This way, Ipe can be extended for each task at hand.
* The text model is based on Unicode, and has been tested with Korean, Chinese, and Japanese.
* The UI is implemented using the portable toolkit Qt, and so can be compiled for Unix, Windows, and Mac OS X (see below).

It can save its figures in the EPS (encapsulated Postscript) format for using with dvips or in the PDF format for using with pdflatex.

Ipe is not very simple to use since it has some special tricks (for example the transition between different modes can be cumbersome sometime), but it is very powerful for drawing even complex figures for LaTeX and it is now my main drawing software.

Introduction

2005-11-13T14:00:00.000+01:00

I start this blog for sharing with you the tools I have discovered during the past years for

writing scientific texts;
drawing figures for LaTeX;
communicating my writings;
keeping my laptop safe and clean;
getting organized with a PDA.

The majority of them are free, some are commercial. I only use commercial software if I cannot find a reliable and efficient open alternative.

I use a laptop under Windows XP as my main computer (I have in the past also used Linux as my main system) and a web server under Linux Mandrake (now called Mandriva).

I hope you will appreciate these tools as much as me...

Murat