Problems with support of file locks for ARC (maybe for other targets too)

[Vineet Gupta]

Hi Yuriy,

On 08/09/2016 10:06 AM, Yuriy Kolerov wrote:
> 1. uClibc does not support OFD locks (https://lwn.net/Articles/586904/). fcntl.h headers for all targets are not synchronized with Linux source tree and does not contain these defines:
>
> ------------------------ 8< ------------------------
> #define F_OFD_GETLK 36
> #define F_OFD_SETLK 37
> #define F_OFD_SETLKW 38
> ------------------------ 8< ------------------------
>
> Also appropriate changes must be done in "libc/sysdeps/linux/common/__syscall_fcntl.c". I am not sure how to do it in the proper manner because it involves a lot of duplication for each target...

This is common generic ABI and applicable only to select few / newer arches.

>
> 2. uClibc uses fcntl64 instead of fcntl by default:
> http://cgit.uclibc-ng.org/cgi/cgit/uclibc-ng.git/tree/libc/sysdeps/linux/common/__syscall_fcntl.c

FWIW - linux/common/* is the common generic syscall ABI that was added to uClibc
when ARC and metag ports were added to linux kernel. The idea is to use
asm-generic kernel headers and for syscalls uses min number of syscalls as
possible (so xxx64 only would be exposed by uapi headers. For this case
specifically, only __NR_fcntl64 will be defied in the kernel.

> Also if you want to use file locks you need to pass flock or flock64 structure to fcntl which are defined in fcntl.h for each target. E.g. for ARC:
> http://cgit.uclibc-ng.org/cgi/cgit/uclibc-ng.git/tree/libc/sysdeps/linux/arc/bits/fcntl.h
>
> So if fcntl64 is used by default then flock must be identical to flock64 in this case. That is why flock and flock64 are defined this way:

Note that given the common generic syscall ABI - userspace doesn't need to use
struct flock64 with flock64 syscall. However the userspace struct flock needs to
match with kernel's struct flock64. See what we do for stat syscall family
(libc/sysdeps/linux/common-generic/bits/stat.h)

> ------------------------ 8< ------------------------
> struct flock
>   {
>     short int l_type;	/* Type of lock: F_RDLCK, F_WRLCK, or F_UNLCK.	*/
>     short int l_whence;	/* Where `l_start' is relative to (like `lseek').  */
> #ifndef __USE_FILE_OFFSET64
>     __off_t l_start;	/* Offset where the lock begins.  */
>     __off_t l_len;	/* Size of the locked area; zero means until EOF.  */
> #else
>     __off64_t l_start;	/* Offset where the lock begins.  */
>     __off64_t l_len;	/* Size of the locked area; zero means until EOF.  */
> #endif
>     __pid_t l_pid;	/* Process holding the lock.  */
>   };
>
> #ifdef __USE_LARGEFILE64
> struct flock64
>   {
>     short int l_type;	/* Type of lock: F_RDLCK, F_WRLCK, or F_UNLCK.	*/
>     short int l_whence;	/* Where `l_start' is relative to (like `lseek').  */
>     __off64_t l_start;	/* Offset where the lock begins.  */
>     __off64_t l_len;	/* Size of the locked area; zero means until EOF.  */
>     __pid_t l_pid;	/* Process holding the lock.  */
>   };
> #endif
> ------------------------ 8< ------------------------
>
> Macros __USE_FILE_OFFSET64 and __USE_LARGEFILE64 are defined in features.h when the compiler have these defines:
>
> ------------------------ 8< ------------------------
> #define _FILE_OFFSET_BITS 64
> #define _LARGEFILE64_SOURCE
> ------------------------ 8< ------------------------
>
> So if you want to use locks you should always define those macros.

Perhaps you know this already: these macros are part of Large File support (LFS).
Traditionally __LARGEFILE64_SOURCE means the 64 based API in explicitly used in
code, while _FILE_OFFSET_BITS=64 means compiler magically converts regular 32 bit
API into 64 bit - under the hood with preprocessor magic.

>  Moreover the whole uClibc and every application must be compiled with those macros. Here is my question. Is it acceptable just to compile toolchain itself using CFLAGS_FOR_TARGET with those macros and don’t define them manually in each application?

If you enable LFS support in build system (uClibc or buildroot) these macros will
be automatically defined - no point or need to pass them as CFLAGS etc - infact
that would be wrong IMO.

So you really don't need LFS to be able to use locks - but need to make sure that
for common generic ABI - user struct flock pairs correctly with kernel struct
flock64 with or w/o LFS.

-Vineet