Solaris CIFS Server and ZFS ACLs: The Problem

I’m going to be switching my home file server over to Solaris soon (or, more specifically, Solaris Express Community Edition [SXCE] build 99), and one of the primary goals of this server is to serve up a few directories to Windows or other SMB clients. One of the reasons I’m switching from Linux to Solaris is because I’m significantly increasing the disk space and I want to use ZFS for my storage pool. At the same time, I’m hoping to take advantage of SXCE’s built-in CIFS server to serve SMB shares.

To prepare for the big switch, I installed the latest build on a test machine and am playing around with setting up a configuration similar to what I’ll want. Unfortunately, it became clear quickly that I was going to hit problems with the new NFSv4 ACLs implemented in ZFS and how the CIFS server interacts with those ACLs on behalf of Windows clients.

So, in this post, I’ll walk through what I want to have happen, and what’s actually happening instead.

I have several users. They all belong to a group named data. There is a directory, /export/sandbox, that is for group project resources for everyone in the data group. All users in the group should be able to create files and directories in sandbox, and everyone else in the data group should be able to modify the files and directories. All users on the system should have read access to the sandbox tree.

This is very easy with traditional Unix permissions. You set the sandbox directory to mode 775, set the group to data, and set the setgid bit. For example, if I’m going to make sandbox a ZFS dataset, I can do:

# zfs create -o casesensitivity=mixed rpool/export/sandbox
# chown root:data /export/sandbox
# chmod 775 /export/sandbox
# chmod g+s /export/sandbox

(The casesensitivity option is to make it play well with Windows as a file share)

Finally, in each user’s profile I can set umask 002 and everything works as desired. Let’s log in as mwilson and do some tests:

$ umask
002
$ cd /export/sandbox
$ ls -l
total 0
$ touch test-file
$ mkdir test-dir
$ ls -l
total 2
drwxrwsr-x   2 mwilson  data           2 Oct  7 17:35 test-dir/
-rw-rw-r--   1 mwilson  data           0 Oct  7 17:35 test-file

Excellent! This is exactly what we wanted: the file’s group is set to data and it remains writable by the group, as does the directory. Both are read-only for the world. The setgid bit on the new directory is set, so this method will work as users continue making subdirectories deeper in the tree.

But now, we’re going to add a new requirement: some users will access the sandbox from Windows using SMB. My server is already set up to run the Solaris CIFS server, so it’s easy to share this folder:

# zfs set sharesmb=name=sandbox rpool/export/sandbox

Now from a Windows client, I can go to \\server\sandbox and sure enough I see the directory and its contents. I’m authenticated as a user that maps to the mwilson Unix user. Now I’ll create a text file from Windows, then look at the directory listing back in Unix:

$ ls -l
total 5
drwxrwsr-x   2 mwilson  data           2 Oct  7 17:35 test-dir/
-rw-rw-r--   1 mwilson  data           0 Oct  7 17:35 test-file
----------+  1 mwilson  data           0 Oct  7 17:47 windows-file.txt

Whoa! Look at the file we created, windows-file.txt. That’s different… the ZFS ACLs are beginning to rear their ugly heads. The + next to the Unix permissions indicates that this file contains extended ACLs. Let’s look at the ACL on this file:

$ ls -v windows-file.txt
----------+  1 mwilson  data           0 Oct  7 17:47 windows-file.txt
     0:user:mwilson:read_data/write_data/append_data/read_xattr/write_xattr
         /execute/delete_child/read_attributes/write_attributes/delete
         /read_acl/write_acl/write_owner/synchronize:allow
     1:group:2147483648:read_data/write_data/append_data/read_xattr
         /write_xattr/execute/delete_child/read_attributes/write_attributes
         /delete/read_acl/write_acl/write_owner/synchronize:allow

Okay. Deep breath. This file has ACL entries that say the user named mwilson is allowed to do, well, just about everything you could ever want to do to the file. The group with ID 2147483648 also has full permissions. Why the weird group number? It’s something to do with the mapping of Windows users and groups to Unix users and groups…honestly, I don’t know where it’s coming from. Since I’m mapped to the mwilson user, I wish it would just apply the Unix user’s group as the effective group if nothing else.

In any case, there seems to be a problem here: the data group no longer has any access to this file! Nor, it seems, does the world have read access.

Let’s log in as another user, jsmith, who is in the data group and look at the sandbox directory.

$ cd /export/sandbox
$ ls -l
./windows-file.txt: Permission denied
total 4
drwxrwsr-x   2 mwilson  data           2 Oct  7 17:35 test-dir
-rw-rw-r--   1 mwilson  data           0 Oct  7 17:35 test-file

Wow… ls gives us an error just trying to list the directory! That’s pretty bad…

Just for kicks, we’ll make a directory from Windows and see what that looks like, as mwilson:

$ ls
total 6
drwxrwsr-x   2 mwilson  data           2 Oct  7 17:35 test-dir/
-rw-rw-r--   1 mwilson  data           0 Oct  7 17:35 test-file
d-----S---+  2 mwilson  data           2 Oct  7 20:45 windows-dir/
----------+  1 mwilson  data           0 Oct  7 17:47 windows-file.txt

Okay, there’s the directory (windows-dir) but it’s again different than what we’re used to. It looks like the Windows file, but has a capital S in the group mode. That indicates that the setgid bit is set, but the execute bit is not set for the group. Let’s check the ACL that’s in place:

$ ls -dv windows-dir
d-----S---+  2 mwilson  data           2 Oct  7 20:45 windows-dir/
     0:user:mwilson:list_directory/read_data/add_file/write_data
         /add_subdirectory/append_data/read_xattr/write_xattr/execute
         /delete_child/read_attributes/write_attributes/delete/read_acl
         /write_acl/write_owner/synchronize:allow
     1:group:2147483648:list_directory/read_data/add_file/write_data
         /add_subdirectory/append_data/read_xattr/write_xattr/execute
         /delete_child/read_attributes/write_attributes/delete/read_acl
         /write_acl/write_owner/synchronize:allow

This is just like the file, but with the directory versions of the permission names instead of the file versions of the names.

Where does that leave us? Creating files and directories from the Unix command line gives me the behavior I want, but creating files from Windows through the SMB share leads to dreadful results.

The problem definitely lies with ACL inheritance: I suspect I will need to define the ACLs that I want on the sandbox directory and set the appropriate inheritance flags, then files created in an “ACL-aware” fashion, such as by the CIFS server, will end up with the permissions I want. We shall see…hopefully soon I’ll have a follow-up post walking through the solution.

Update: as if it isn’t obvious by now, I never did follow up and figure out the right ACLs. I ended up using Solaris 10 Update 6, instead of OpenSolaris, to build the new file server. Solaris doesn’t have the integrated CIFS server yet, so I’m going the traditional Samba route, with doesn’t have this problem.

Leave a comment ?

12 Comments.

  1. Take a look at Afshin’s post here:

    http://opensolaris.org/jive/thread.jspa?messageID=278848

    From that it wasn’t hard to figure out how to set up something very similar to what you’re trying to achieve.

  2. OpenSolaris and CIFS share ACLs are bit hard to deal with initially. The problem starts with the fact that ZFS shares do not come with inheritable ACL permissions out of the box – and both Windows and OS X (I’m dealing with Leopard) clients only pay attention to the ACLs and ignore umask and other UNIX based permissions.

    To fix this issue, you simply need to create a standard set of ACLs with inheritance marked on the directory you are sharing, something like this (careful this will replace all existing ACLs):

    chmod -R A=owner@:rwxpdDaARWcCos:df:allow,everyone@:xrcarR:df:allow

    This gives a new file with permissions like this:
    user:full control
    everyone: read_only (it includes reading of child objects and attrs, etc).

    The moral of the story is to define a default set of ACLs when you create a new ZFS/CIFS share.
    (Most of this was derived from the post by Afshin: http://opensolaris.org/jive/thread.jspa?messageID=278848 — Mentioned by Kopsis in another comment).

  3. Sorry for the additional comment – but the comment box stripped off the last bit of the chmod command. You obviously need to add the directory name to the end of the command – the directory you want to apply the ACL to.

    Hope this helps anybody out there that is looking for this information.

  4. Indeed. Thanks, Bryan.

  5. I can’t get Bryan’s chmod command to work. I’m on OpenSolaris JeOS svn_134 and issuing:

    chmod -R A=owner@:rwxpdDaARWcCos:df:allow,everyone@:xrcarR:df:allow /export/home/cfissahre/osol

    results in the following error:

    chmod: invalid mode: ‘A=owner@:rwxpdDaARWcCos:df:allow,everyone@:xrcarR:df:allow’

    What am I missing?
    Thanks

  6. @Benjamin – the command you list works fine for me on Solaris 10u8. On OpenSolaris I think there’s a possibility that the GNU chmod command is higher on your path than the Solaris chmod command. What does ‘which chmod’ tell you? Try your command again, but specify a full path for chmod — /usr/bin/chmod should work.

  7. @mwilson
    That did it! Thanks a bunch!

  8. I got the answer you are looking for on my Solaris 11 guide. Check out under “Permissions and ACL”.

  9. Thank you.

  10. I’m using Nexentastor which is built on OpenSolaris 134f.

    /usr/sun/bin/chmod -R A=owner@:rwxpdDaARWcCos:df:allow,everyone@:xrcarR:df:allow /volumes/pool/cifs
    chmod: ERROR: Failed to set ACL: Operation not applicable
    chmod: ERROR: Failed to set ACL: Operation not supported
    chmod: ERROR: Failed to set ACL: Operation not supported

    /usr/bin/chmod -R A=owner@:rwxpdDaARWcCos:df:allow,everyone@:xrcarR:df:allow cifs
    /usr/bin/chmod: invalid mode: `A=owner@:rwxpdDaARWcCos:df:allow,everyone@:xrcarR:df:allow’

    Please, any response to my email address please. I appreciate any help.

    Ron

  11. I hope this command will help you all
    # chmod -R A+group@:full_set:fd:allow /cifsshare

  12. Your problem is that you are expecting solaris to force windows shares to act as if they have unix permissions wheras it is making unix serve up windows friendly permissions. In a windows environment it is the samba way that is horrible.

    Neither way is wrong, they are just suited to different situations. You should look at it that it is trying to serve up each type of share in the way the clients like, not as a transparent way to bridge two fundamentally different approaches to permissions.

Leave a Comment


NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>