Moving to https://nantipov.org

Hello,

I am moving to https://nantipov.org .

See you.

iumls

I've just started my new tiny open-source project.

http://code.google.com/p/iumls/

how to find large and last modified files on linux?

I decided to make a short note about how to find file which were recently modifed and also they have a quite big size.

It's to find the cause of disk overfill.


find /var -type f -size +100M -exec stat --format '%Y :%y %n' {} \; | sort -nr | cut -d: -f2- | head

solution has been got from http://stackoverflow.com/questions/5566310/how-to-recursively-find-and-list-the-latest-modified-files-in-a-directory-with-s .

Java, Trac, RPC and JSON

I thought it would be great (and maybe useful for you) to publish here my experience how to set up Trac with RPC over JSON. When I tried to do it from the scratch I had realized that a lot of things work in unexpected way, at least for me.

The main goal is to manage Trac tickets with some application. Here you'll get all information about plugin http://trac-hacks.org/wiki/XmlRpcPlugin. I want to make some details clear for you and show you my ready/worked code.

Trac and Apache

Ok. Short list of items.

• install trac;
• install plugin 'TracXMLRPC';
• enable plugin (tracrpc.* = enabled in trac.ini, see below);
• set up web server with authorization;
• create user with 'XML_RPC' privileges in trac;
• it's really simple, use Basic Authorization (see below green pieces of text);
• create user ('htpasswd') and grant its to 'XML_PRC' (

  trac-admin /path/to/projenv permission add user_name XML_PRC

  );
• Also, you maybe would be interested to set up mixed Authorization (fixed files + LDAP), see orange pieces of text below);
• In Java (see below):
• do auth;
• create JSON;
• send and interpret.

Apache configuration

       

<IfModule wsgi_module>
        WSGIScriptAlias /trac /path_to_trac/trac_site/cgi-bin/trac.wsgi
        WSGIPassAuthorization On
        <Directory /path_to_trac/trac_site/cgi-bin>
                WSGIApplicationGroup %{GLOBAL}
                Require all granted
                AllowOverride None
                Order allow,deny
                Allow from all
        </Directory>
        <Location /commref_req_trac/login>
                AuthBasicProvider file ldap
                AuthType Basic
                #AuthzLDAPAuthoritative off
                AuthUserFile /path_to_trac/trac_env/passwd_file
                AuthName "Trac"
                AuthLDAPURL "ldap://domain.com:389/dc=domain,dc=com?sAMAccountName?sub?(objectClass=user)"
                AuthLDAPBindDN "CN=user_ldap,OU=Service Accounts,OU=GROUP,DC=domain,DC=com"
                AuthLDAPBindPassword "user_ldap_password"
                Require valid-user
        </Location>
</IfModule>

       

Trac configuration

[components]
tracrpc.* = enabled

Java

Transport

How to interact with Trac. For JSON I used  library 'org.json' ('https://github.com/douglascrockford/JSON-java') and 'Apache Commons Codec' ('http://commons.apache.org/proper/commons-codec/') for Base64.

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.net.URL;
import java.net.URLConnection;

import org.apache.commons.codec.binary.Base64;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;

public class TracAPI {

    private static int jsonIdSequence = 1;

    public static JSONObject sendJsonRpc(String method, JSONArray parameters) throws IOException, JSONException {
        JSONObject rpc = new JSONObject();
        rpc.put("method", method);
        rpc.put("params", parameters);
        rpc.put("id", jsonIdSequence++);

        String tracUrl = Configuration.getTracApiUrl();
        String tracUsername = Configuration.getTracApiUsername();
        String tracPassword = Configuration.getTracApiPassword();

        URL url = new URL(tracUrl);
        URLConnection conn = url.openConnection();
        String userpass = tracUsername + ":" + tracPassword;
        String basicAuth = "Basic " + new String(new Base64().encode(userpass.getBytes()));
        conn.setRequestProperty ("Authorization", basicAuth);
        conn.setRequestProperty ("Content-Type", "application/json");
        conn.setDoOutput(true);

        OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());

        Logger.debug("jsonrpc request:\n" + rpc.toString());

        wr.write(rpc.toString());
        wr.flush();
        wr.close();

        BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
        String line;
        String jsonText = "";
        while ((line = rd.readLine()) != null) {
            jsonText += line;
        }
        rd.close();

        Logger.debug("jsonrpc response:\n" + jsonText);

        return new JSONObject(jsonText);
    }

}

Payload

 

read ticket

                    JSONArray parameters = new JSONArray();
                    parameters.put(ticketId); //id
                    JSONObject resp = TracAPI.sendJsonRpc("ticket.get", parameters);
                    String status = resp.getJSONArray("result").getJSONObject(3).getString("status");
                    String resolution = resp.getJSONArray("result").getJSONObject(3).getString("resolution");
                    String owner = resp.getJSONArray("result").getJSONObject(3).getString("owner");

create ticket

                JSONArray parameters = new JSONArray();
                parameters.put("summary text"); //summary
                parameters.put("description text"); //description
                parameters.put(new JSONObject()); //attributes
                parameters.put(false); //notify

                JSONObject resp = TracAPI.sendJsonRpc("ticket.create", parameters);

update/modify ticket

Update must be performed in two stages:

• Determine (current) version (timestamp) of ticket (to protect ticket from concurrent modifications, you have to indicate which version of ticket you're modifying);
• Perform modification.

Determine timestamp.

            JSONArray parameters = new JSONArray();
            parameters.put(ticketId); //id
            JSONObject resp = TracAPI.sendJsonRpc("ticket.get", parameters);
            long ts = resp.getJSONArray("result").getJSONObject(3).getLong("_ts");

Update.

            JSONObject attrs = new JSONObject();
            attrs.put("action", "accept");
            attrs.put("_ts", ts);

            parameters = new JSONArray();
            parameters.put(ticketId); // id
            parameters.put("comment text"); // comment
            parameters.put(attrs); // attributes
            parameters.put(false); // notify
            resp = TracAPI.sendJsonRpc("ticket.update", parameters);

Feel free to write comments if you have questions/suggestions or another stuff you'd like to share. Thank you.

Java code was highlighted with "http://markup.su/highlighter/".

LDAP/AD user password check

Simple algorithm.

1. connect to ldap;
2. auth with "super"-user (user1): special user with known DN and password;
3. search another user (user2): user who must be tested, search him with some attribute (for example sAMAccountName);
4. determine DN of user;
5. open one more connection to ldap with just found DN and password of user2;
6. no exception - ok, otherwise user2 has entered wrong password.

My code:

/**
 * Returns {@link Item} with addition information in case of user exists and authicated correctly, otherwise null
 * @param username username
 * @param password password
 * @return Item or null
 */
private Item checkWithLdap(String username, String password) {
    String ldap = CXConfiguration.getParameterAsString("LDAP_SERVER");
    if ((ldap != null) && !ldap.isEmpty()) {
        DirContext ctx = null;
        try {
            int port = CXConfiguration.getParameterAsInt("LDAP_SERVER");
            String ldapUrl = "ldap://" + ldap + ":" + port;
            Logger.info("checking with ldap [username: " +
                    username + ", password: ******, ldap: " +
                    ldapUrl + "]");
            Properties env = new Properties();
            env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
            env.put(Context.PROVIDER_URL, ldapUrl);
            env.put(Context.SECURITY_AUTHENTICATION, "simple");
            env.put(
                    Context.SECURITY_PRINCIPAL,
                    CXConfiguration.getParameterAsString("LDAP_AUTH_DN")
            );
            env.put(
                    Context.SECURITY_CREDENTIALS,
                    CXConfiguration.getParameterAsString("LDAP_AUTH_PASSWORD")
            );
            ctx = new InitialDirContext(env);
            String base = CXConfiguration.getParameterAsString("LDAP_BASE_DN");
            String[] attributeFilter = {
                    CXConfiguration.getParameterAsString("LDAP_USER_SEARCH_ATTR"),
                    "distinguishedName",
                    CXConfiguration.getParameterAsString("LDAP_EMAIL_ATTR")
            };
            SearchControls sc = new SearchControls();
            sc.setReturningAttributes(attributeFilter);
            sc.setSearchScope(SearchControls.SUBTREE_SCOPE);
            String filter = "(" +
                    CXConfiguration.getParameterAsString("LDAP_USER_SEARCH_ATTR") +
                    "=" + username + ")";
            NamingEnumeration<SearchResult> results = ctx.search(base, filter, sc);
            if (results.hasMore()) {
                Properties env1 = new Properties();
                env1.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
                env1.put(Context.PROVIDER_URL, ldapUrl);
                SearchResult result = (SearchResult) results.next();
                Attributes attrs = result.getAttributes();
                Attribute dnAttr = attrs.get("distinguishedName");
                String dn = (String) dnAttr.get();
                Attribute emailAttr =
                        attrs.get(CXConfiguration.getParameterAsString("LDAP_EMAIL_ATTR"));
                String email = (emailAttr.get() != null) ? (String) emailAttr.get() : "";
                env1.put(Context.SECURITY_PRINCIPAL, dn);
                env1.put(Context.SECURITY_CREDENTIALS, password);
                new InitialDirContext(env1); // in case of problem exception will be threw
                Item i = new Item();
                i.setValue("email", email);
                Logger.info("checking with ldap - ok");
                return i;
            } else return null;
        } catch (Exception e) {
            Logger.error("checking with ldap - ldap error", e);
            return null;
        } finally {
            if (ctx != null)
                try {
                    ctx.close();
                } catch (NamingException e) {
                }
        }
    } else return null;
}

highlighted with: http://tohtml.com/

Хорошая статья "Безопастность в Java"

Хорошая статья на русском языке о безопастности в Java. В двух словах, доступно, просто и понятно. Отличная статья для старта обзора этой области.

 http://voituk.kiev.ua/2008/08/18/bezopastnost-v-java/

GSM 7bit (part of PDU) pack/encoding algorithm in Java

GSM 7bit (part of PDU) pack/encoding algorithm in Java. This can be used for SMS/USSD writing.

public static String stringTo7bit(String string) {
        String hex = "";
        byte[] bytes = string.getBytes();
        int f = 0;
        while (f < bytes.length - 1) {
            int t = (f % 8) + 1;
            if (t < 8) {
                byte b = (byte) (((bytes[f] >>> (t - 1)) |
                    (bytes[f + 1] << (8 - t))) & 0x000000FF);
                hex += intToHex(b & 0x000000FF).toUpperCase();
            }
            f++;
        }
        if ((f % 8) + 1 < 8)
            hex += intToHex((bytes[f] >>> (f % 8)) & 0x000000FF).toUpperCase();
        return hex;
    }

public static String intToHex(int i) {
        String hex = Integer.toHexString(i);
        if (hex.length() % 2 != 0) hex = 0 + hex;
        return hex;
    }

highlighted with: http://tohtml.com/