Monday, July 30, 2012

Installing Glusterfs 3.3 on OpenIndiana 151

[This is still under testing. I did based on XinFeng Liu's installation on Solaris 11. The Glusterfs 3.3.0 worked well on OpenIndiana 151a5 x64, but might need some fine tuning by others. Comments are welcomed!]

I started with new clean installation of OpenIndiana 151a.
Login as root and update the system to the latest version,
su -
pfexec pkg image-update -v
(All proceeding commands assume that you are root.)

Add SFE repository,
pkg set-publisher -p
pkg set-publisher -p
If you have the legacy gcc-4 runtime from the publisher installed, you will run into trouble. Uninstall it by running,
pkg uninstall pkg://* 
Install gcc-dev, libfuse, ntfs-3g,
pkg install gcc-dev 
pkg install libfuse
pkg install ntfs-3g
Download Glusterfs 3.3.0 sources,
Untar it,
tar -xvzf  glusterfs-3.3.0.tar.gz

Need to perform the following patch otherwise compilation will fail,
(all the commands assume that you are in the extracted glusterfs-3.3.0 directory)

1) nano rpc/xdr/src/nlm4-xdr.h
Add the following 2 lines:
#define  u_int32_t       uint32_t
#define  u_int64_t       uint64_t

2) nano ./libglusterfs/src/compat.h
Add the following patch at the end of the file:
/* This patch is not present in Solaris 10 and before */
#ifndef dirfd
#define dirfd(dirp)   ((dirp)->dd_fd)
Solaris 11 uses POSIX definition for "DIR"

3) nano cli/src/cli-cmd-volume.c
In cli_get_detail_status (), add the following in between each of Linux-related attributes for "device", "mount_options", "fs_name" and "inode_size". (The #ifdef and #endif is already there for "device". Follow the same format for other attributes.)


Now, things are ready for compilation. We start with configure,
CFLAGS=-m64 ./configure

Need pass -m64 in CFLAGS above, or otherwise libtool will use 32-bit and you will end up with error during gmake.

After patching the source code as in 1) until 3) above, gmake should proceed until it finishes (without error).

gmake install

Installation should complete properly, with executable files installed in /usr/local/sbin directory.

Lets's check the installation,
/usr/local/sbin/glusterfs --version
glusterfs 3.3.0 built on Jul 29 2012 21:08:18
Repository revision: git://
Copyright (c) 2006-2011 Gluster Inc.
GlusterFS comes with ABSOLUTELY NO WARRANTY. You may redistribute copies of GlusterFS under the terms of the GNU General Public License. 
Let's play around with glusterfs,
cd /usr/local/bin/ 

Create a volume with name "test-volume" on server with hostname "storage0":
gluster> volume create test-volume storage0:/test-vol
Creation of volume test-volume has been successful. Please start the volume to access data.

gluster> volume list 

gluster> peer status  
No peers present 

gluster> peer probe storage0  
Probe on localhost not needed 

gluster> volume info  
Volume Name: test-volume Type: Distribute Volume ID: b1cee565-c940-4ac5-8d66-a449518f5dfd Status: Created Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: storage0:/test-vol 

gluster> volume start test-volume  
Starting volume test-volume has been successful 

Seems like things are working well. Alhamdulillah :)



Anonymous said...

Hi, have you tried 3.3.1 release? It seems that it does not compile correctly following your guidelines.

It stops saying "d_type" not found...

Please, can you confirm it?

Best Regards,

Anonymous said...

Could you make #3 (the modifications to cli/src/cli-cmd-volume.c) more clear? I'm not sure exactly how to modify this file from the instructions.

Thanks again for the great post!

Anonymous said...


> It stops saying "d_type" not found...

I found a solution...
So i have written a copy&paste friendly make howto, with all known patches.

regards heiko