How to su with a here doc

Not sure why, but it took me ages to work out the correct syntax to run su with  a shell and a here document.

Turns out the correct syntax, or at least syntax that seems to work, is:

su - <user> -c /bin/bash <<-'EOF'
<some commands...>
EOF

Now this assumes we're running as root, else you'd need to enter a password.
EOF in the above example is just a delimter user to indicate where the here doc ends , it can be more or less any string, but I'd keep it alpha to keep things simple, i.e. avoid punctuation and the like.

e.g.

[root@clivm ~]# cat test.sh
#!/bin/sh

#run the 'id' command to show we're running as root
id
su - fred -c /bin/bash <<-'MEH'
#now run id to show we're runnning as fred and some echos 
#to show where the user 'fred' session starts and ends
# fred's session starts and ends
echo "---running as ${USER}--"
id
echo "now I am fred:"
echo
echo "---about to hit the end delimiter and exit ${USER}'s context"
MEH
# show that we have left fred's context and are running as  root again
id
echo "---running as ${USER}--"

The above example uses MEH as the delimiter.
When run as root it looks like this:
The bold section is executing as user fred.

[root@clivm ~]# sh test.sh
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
---running as fred--
uid=500(fred) gid=500(fred) groups=500(fred) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
now I am fred:

---about to hit the end delimiter and exit fred's context
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
---running as root--
[root@clivm ~]#

Network Interfaces And The "ip" Command 

With the release of RHEL 7 the long threatened removal of the ifconfig has finally come about.
Ran into my first RHEL7 (Centos7 actually) box today while patching some VMs and had to work out the dreaded ip command.

In  a nutshell
Assuming you have a machine that needs it's network configured nad it does not have any configured interface  and you want to use the following details:
ip 10.10.10.23
gateway 10.10.10.1
netmask 255.255.255.0
network device: ens32

You can probably do it with these commands:

ip addr add 10.10.10.23/255.255.255.0 dev ens32
ip route add default via 10.10.10.1 dev ens32

View addresses with:
ip addr show

and routes with 
ip route show

completely unsurprisingly deleting is...
ip addr del 10.10.10.23/255.255.255.0 dev ens32
ip route del default via 10.10.10.1 dev ens32

Anything cleverer is left as an exercise for the reader, because at this point in time I probably have NFI.

Hope it helps.
If not ask me a question. I might be able to answer it.

More GNU date magic

More GNU Date Magic

Many unix users don't realise that the  GNU date utility can be used for far more than simply getting and setting the time.
GNU date is a very versatile and handy utility for calculating, formatting and manipulating dates generally.

One of the things you need to know about that is that you can set the time and date that you want date to work on.

Basically there are two dates that date works with:

• now i.e. the time and date of execution or
• some other arbitrary date specified by the caller

To use now you often will not need to specify anything

e.g.
[jtan@clivm 0509]$ date
Mon Sep  1 11:50:28 CST 2014 

returns the time and date that you executed the command.
In this case 11:50 and 28 seconds on Monday Sep 1 2014 CST (Central Standard Time)

But if I want to know the date for 60 days in the future and I am too lazy too count the days in the calendar or I want to use that in a script I can tell date to print that time.

The way you tell date to do it's work based on a time other than now is to use the -d option.

Now the real date magic is that the -d option is very flexible in the format of the options it will accept.

e.g. to find out the date and time 60 days from when I run the command I can run:
 date -d  "now +  60 days"
and it will give me
[jtan@clivm 0509]$ date -d "now + 60 days"
Fri Oct 31 12:55:48 CST 2014

Now I have used the keyword now to indicate that I want date to run using the time of execution as it's base time to use.

However that is actually redundant in this case, because now is actually the default, so date -d "+60 days" will also work. In fact date -d"60 days" will also work, which I think is about as minimal as you can go and still express the desired result.

You can of course also pass date a format in the standard manner:

[jtan@clivm 0509]$ date -d "now 1 month"  "+%d-%m-%Y"

01-10-2014

Postgres's psql and schemas

When I was a lad and the ancestors of the dinosaurs had just emerged from the primordial soup, you could run the psql client with a named data base and just type something like:
select something, something_else, a_third_thing from some table_or_other;

And get a result.
Then along came schemas.
What's a schema? It's a namespace that allows tables within a database to be grouped.
Now when you run the above you'd get something like:

ERROR:  column "something" does not exist
LINE 1: select something, something_else, a_third_thing from some_table_or_other;

This is because you need to specify the schema you want to search.
How do you know what schema to search?
Well you can list the tables and Postgres will by default show a column with the schema names, e.g.:

ckan_prd=# \dt

                     List of relations

 Schema |             Name              | Type  |  Owner

--------+-------------------------------+-------+----------

 public | activity                      | table | ckan_prd

 public | activity_detail               | table | ckan_prd

 public | authorization_group           | table | ckan_prd

 . . .

You can also list the schemas themselves e.g :

ckan_prd=# \ds

                     List of relations

 Schema |            Name             |   Type   |  Owner

--------+-----------------------------+----------+----------

 public | system_info_id_seq          | sequence | ckan_prd

 public | system_info_revision_id_seq | sequence | ckan_prd

(2 rows)

. . .

Once you have identified the schema there are a couple of ways you can specify it, that I know of.

First you can do it in the query, by prepending the relevant schema to the table name(s)* with a dot ('.'), e.g.

select something, something_else, a_third_thing from some_schema.some_table_or_other

This may seem a little counter-intuitive as the error message above seems to indicate that postgres is havnig a problem finding the column, not the table. But no matter. I'm sure if you think about it there is a perfectly logical explanation. I've declined to think about it, so I can't tell you. Anyhoo...

The other way you can do it, if you know you're going to be working with a single schema all or most of the time is set the SCHEMA environment variable, e.g.:

set SCHEMA 'public';

Note: the variable name SCHEMA is case sensitive, the value 'public' is single quoted and the end of statement semi colon (';') is required.
That's it. Go forth and query postgres.

* Note: not actually sure if you can query tables from different schemas in a single query. I assume you can but I'm not really an RDBMS man.

References:

• Postgres 9.4 Schema Docco
• Postgres 9.4 Environment Variable Docco

A little date magic

Convert A Unix Timestamp to Human Readable Localtime

Note: the version of date used here is GNU date, that is supplied with most free unices. It may also be supplied with some proprietary unices as an optional, third party or perhaps standard option. GNU date is far more powerful and featureful than most of the proprietary date commands I've experienced, so if you're on a system without GNU date, YMMV.

Unix time is stored in seconds since 00:00:00 1 Jan 1970 (it may actually be 00:00:01 1 Jan 1970, not 100% sure and it doesn't make much difference for most operations).
00:00:00 1 Jan 1970 is referred to as the Unix epoch, the time unix started to exist.
This method of  measuring time is unsurprisingly  referred to as "Unix epoch time" or "Unix time". Less accurately (because there is more than one kind of system) "system time".

Unix Epoch Time to Human Readable System Local

To take seconds since 00:00 1 Jan 1970 (i.e. unix epoch time) and convert it to the current system local time:
date -d @<seconds> since 00:00 1 Jan 1970>

e.g.:
[root@myhost mydir]# date -d @1408329090 
Mon Aug 18 12:01:30 CST 2014

System Local Time as Unix Epoch Time

To print the local system time in Unix epoch time:
date +%s

UMT

[root@ckan-db .ssh]# date
Mon Aug 18 12:44:39 CST 2014
[root@ckan-db .ssh]# date -u
Mon Aug 18 03:14:41 UTC 2014
[root@ckan-db .ssh]#

Notes on persisting SELinux changes

Some notes on SELinux, and in particular persistence of changes over reboots.

Some of the anecdotal issues mentioned occurred a couple of years ago and may be resolved now, however in my mind it is not unreasonable to assume they are still current - you won't lose anything apart from a little time by assuming they are still current.

Identifying context

ls, ps netstat have a -Z option. tar supports SELinux however -Z was the compress flag in tar, before SELinux was a twinkle in some anonymous NSA nerd's eye, so the -Z option on tar won't get you SELinux info. --selinux and --noselinux are the relevant tar options. This will show SELinux info, such as SELinux user and file contexts. Obviously as always, man is your friend.

Don't trust the logs

SELinux logs to the audit.log file which is usually found at /var/log/audit/audit.log on a RHEL based host. When it logs.
In the past we've seen issues where an app would not work as desired but no SELinux event was logged. However turning SELinux off resolved the issue.
Conclusion? At that point in time not all SELinux events were logged. I'm not sure if this is still the case but if I have a problem that I can't identify and SELinux is enforcing, then I'll turn it off just to make sure.

Permissive mode can break things

Another blast from the past. Under RHEL5 running SELinux in permissive (i.e. log violations but don't enforce them) has been cited as a possible cause of issues for very time sensitive operations. In particular Oracle have advised that merely running SELinux in permissive mode can introduce enough additional latency to cause issues with active/active clusters. Again I am not sure if this is still the case, but it makes sense. If SELinux is permissive or enforcing every system call made is checked against the current state of the SELinux context and that will take some time and it's possible that extra time will introduce some additional latency and consume some number of instructions and possibly cause context changes and maybe pipeline purges.

Persistance

The default's in the current SELinux policy are applied on boot/reboot. This is sometimes referred to as "relabelling".

Temporary changes (e.g. chcon, setsebool) are lost on reboot as the policy is re-applied.

To edit the policy and therefore persist changes you need to use semanage. 

Notes and examples below.

getsebool -a|grep http   # list all booleans current state - handy to see if you can see something that looks appropriate to what you are doing.

#this changes the current state of the boolean but I don't think will persist across a reboot ie. is useful for testing and trouble shooting.

sesetbool httpd_can_network_connect_db=1

semanage allows you to manipulate the current SE Linux policy - therefore changes made with semanage  will persist across reboot.

semanage boolean -l

# will also list all booleans  current state  as does getsebool, however semanage boolean will also list the booleans default state as defined in the policy (i.e. what they will be set to on reboot)  and a description of their function.

e.g.:

semanage boolean -l:| grep httpd_can_network_connect

httpd_can_network_connect      (on   ,  off)  Allow HTTPD scripts and modules to connect to the network using TCP.

The "on" is the current state, the "off" is the policy's current default so the httpd_can_network_connect booelan would revert on reboot).

semanage boolean --on httpd_can_network_connect

Sets current state AND the policy's default - this should persist over reboots.

#check the policy default has changed

semanage boolean -l |grep httpd_can_network_connect

httpd_can_network_connect      (on   ,   on)  Allow HTTPD scripts and modules to connect to the network using TCP.

Now the SELinux policy's  default as well as the current state is changed to on. I believe this will persist accross boots.

You get the same behavior with chcon the command for changing file contexts .

I.e. changes made with chcon will not persist as they only occur on the filesystem which is "re-labelled" on boot to the defaults in the policy

So if you change a file context with chcon that change probably won't persists across reboots.

The file will be "re-labelled" when the SELinux policy as applied at boot.

chcon is handy for investigation and one off temporary fixes though.

You need to use semange fcontext to alter file contexts in the policy.

Pretty much audit2allow, semodule (which used to be how you loaded the generated module from audit2allow) and setting file contexts (chcon) and booleans (setsebool) are all the SELinux I've ever had to do to get stuff running with SELinux. I've either got it running with those three constructs or given up and used the ultimate SELinux fixer upper tool - banish ye SELinux. setenforce 0 !!

Using AWK

awk is a script interpreter designed to process tuples (records) of data structured into columnar fields.
Probably the most common awk implementation today is GNU's gawk found in most linux distros.

Awk is block structured and an awk script takes the general form:

[BEGIN{<commands to execute before processing records>}]{<commands to execute for each record>}[END{<commands to execute after all records are processed>}]

Summing values...
Assuming <somefile> has a number as it's second awk parseable field the following should sum those values from each record in the file.
cat <somefile> | awk '{size=size+$2}END {print size}'