Halloween credit aggregates

It is the last day of the month (how this year has gone by). Which means another round of credit aggregates from the Reserve Bank of Australia. The headline statistic is ongoing but subdued growth (with a flicker of improvement in the month-on-month growth for September).

Writing functions in R

Another cheat sheat! It is the last one in the folder so you will need to scroll down.

Functions is where R deviates from its C-C++-Java look and feel with curly brackets and the like, to reveal its Lisp roots. 

CPI - gas and electricity

CPI Charts: Electricity at 15 per cent for the quarter

The shock result was that electricity prices increased by 15 per cent for the September quarter. Fruit and vegetables increased at 10 per cent over the quarter.

The headline growth for the quarter was also higher than expected. Given the last two quarters, I suspect the Reserve Bank has reduced head room for further rate cuts.

The analyticals follow ... a fair number of the analytical indicators were over one per cent for the quarter.

And Sydney is this quarter's hot spot ...

Finally, non-tradable inflation rate remains high (and the September quarterly result was particularly high). The tradable rate has now seen two positive quarters.

MYEFO 2012

In considering this year's Mid-Year Economic and Fiscal Outlook statement from the Government, it is worth thinking about the government's forecast performance since the global financial crisis.

First, however, let's look at the fiscal policy headlines. The revenue projection for the current financial year is down by $2.4b, and projected expenses are down by $1.4b. The projected fiscal balance is down from $2.5b to $1.2b.

In cash terms: receipt projections are down $1.8b. Payments are down by $1b. The projected underlying cash balance is down from $1.5b to $1.1b.

Now back to looking at the performance of past forecasts. For expenses, since 2007, there is a long-standing tradition of underestimation.

For revenue, with the exception of UEFO and the Budget in 2009, there is a long-standing tradition of overestimation.

The balance projections have not much moved.

Avoiding for-loops - yet another R cheat sheet

[link]

Another R Cheat Sheet: basic list of useful functions

Another in my series of cheat sheets on the R programming language.

This one provides a basic list of useful functions.

The next one will be on the "apply" family of functions.

Car-boom

The ABS motor vehicle sales for September was released today.

More on GDP, GDI and Okun

In the comments to an earlier post, Andrew Lilley suggested that I look at a one or two quarter lag to GDP. He throught I would find a stronger relationship with lagged data.

With a little rewriting of my R program (which I have posted below), I have compared the lagged data. But first I should explain a couple of other changes. in my initial model I compared the quarterly GDP/GDI figure with the first difference of the UR data for the last month in the quarter. In updating my programs, I am now taking the mean of the UR rate for the three months of the quarter.

The second thing to note is that I have checked the UR first difference data from leading by three quarters (a negative lag) to laging by three quarters.

The third thing to note is that I have added a Bayesian Information Criterion as the comparator, rather than R-squared (which is at best problematic as an information criterion). This turns out to be significant in the selection of the winning model (given the R-squared results.

The conclusion:  The GDI growth rare remains better than the GDP growth rate at explaining changes the unemployment rate (selecting the requression with the lowest Bayesian Information Criterion). It does its best job when not lagged. The GDP growth rate (Quarter on Quarter) does its best job when it is lagged by two quarters.

delta(UR)@t = GDPgrowth@t-2 + u

(comparing this quarter with the previous quarter)

The GDP growth rate on a through the year basis does its best job when lagged by one quarter.

delta(UR)@t = GDPgrowth@t-1 + u

(comparing this quarter with the same quarter last year)

The charts on the winning regression models follow.

The R code follows. Please let me know if you see any errors.

### r-okun.R
###
###     GNU R batch file for plotting graphs
###

### ----- create the right environment ...
source("../bin/mysqlConnect.R")
source('../bin/i-plot-constants.R')
library('zoo')
library('plyr')
imageDir <- paste(imageDir, 'okun/', sep='')

### ----- home brew lagging function 
lagger <- function(vector, k)
{
 # - sanity checks
 k <- as.integer(k)
 n <- length(vector)
 if(n == 0) return (NULL)
 if(k > n)  return (NULL)
 
 # - do the lag - pad with NAs
 lagged = NULL
 if(k == 0) lagged <- vector
 
 if(k > 0)
 {
  lagged <- c(rep(NA, k), vector)
  lagged <- lagged[1:n]
 }
 
 if(k < 0)
 {
  k <- abs(k)
  lagged <- c(vector, rep(NA, k))
  lagged <- lagged[(1+k):(n+k)]
 }
 
 return (lagged)
}

### --- get National Accounts from MySQL database
rs <- dbSendQuery(conn, "select * from TT_5206001_key_aggregates_Data1")
natAccounts <- fetch(rs, n = -1)

### --- get Monthly Labour Force (MLF) survey data rfom MySQL database
rs <- dbSendQuery(conn, "select * from TT_6202002_Data1")
seasAdjEmploy <- fetch(rs, n = -1)

### ----- calculate the quarterly mean unemployment rate
endDate <- as.Date(as.yearmon(seasAdjEmploy$t), frac=1.0)
seasAdjEmploy$Qtr <- factor( as.yearqtr(endDate) )
seasAdjEmploy <- ddply(seasAdjEmploy, "Qtr", transform,
 mean.UR.for.qtr = mean(A181525X))

### ----- merge National Accounts & MLF - only keep EOQ MLF records
mergedData <- merge(natAccounts, seasAdjEmploy, by = "t", all = FALSE)

### ----- keep needed columns from the merged data.frame - add Date
# A2304402X ==> GDP: CVM ; $ Millions : Seasonally Adjusted
# A2304410X ==> Real GDI: CVM ; $ Millions : Seasonally Adjusted
# A181525X --> SA_Unemployment_rate_Persons
df <- mergedData[, c('t', 'A2304402X', 'A2304410X',
 'A181525X', 'Qtr', 'mean.UR.for.qtr')]
endDate <- as.Date(as.yearmon(df$t), frac=1.0)
df$Date <- as.Date(as.yearqtr(endDate), frac=0.5)

### ----- do the regressions and plots 
for(period in c('a', 'q'))
{
 freq <- 0
 if(period == 'a') { freq <- 4; p <- 'TTY' }
 if(period == 'q') { freq <- 1; p <- 'Quarterly' }
  
 GDI.growth.percent.name <- paste(period, 'GDI.growth.percent', sep='')
 GDP.growth.percent.name <- paste(period, 'GDP.growth.percent', sep='')
 UR.diff.name <- paste(period, 'UR.diff', sep='')

 df[, GDI.growth.percent.name] <- calculatePercentGrowth(v=df$A2304410X, freq=freq)
 df[, GDP.growth.percent.name] <- calculatePercentGrowth(v=df$A2304402X, freq=freq)
 df[, UR.diff.name] <- calculateDifference(v=df$A181525X, freq=freq)
 
 # - an initial plot of the data ...
 if(period == 'a')
 {
  dlm <- df[!is.na(df[, GDI.growth.percent.name]), ]
  breaks <- c(GDP.growth.percent.name, GDI.growth.percent.name,
   UR.diff.name)
  labels <- c('GDP Percent Growth',  'GDI Percent Growth',
   'Change in Unemployment Rate (% points)')
  gf <- data.frame(breaks=breaks, labels=labels)
  f <- paste(imageDir ,"cross-check-gdi-gdp-ur-tty-",  sep="")
  chartMulti (df=dlm, graphFrame=gf, saveLocPrefix=f,
      plotText=list(heading='TTY Change in GDP, GDI and the Unemployment Rate',
       yLabel='Percent/Percentage Points'), 
      footnote='Source: ABS 5206 and 6202' )
    }

 for(lag in -3:3)
 {
  l <- as.character(lag)
 
  # - lag the differenced MLF data
  column.name = paste('UR.diff.lagged.', l, sep='')
  column.name = sub('-', 'minus.', column.name)
  df[, column.name] <- lagger(df[, UR.diff.name], lag)
   
  print('-------------------------------------------')
  cat(lag); print(' <== unemployment rate 1st difference lagged by ')
  print('-------------------------------------------')

  for(fn in c('P', 'I'))
  {
   # - set up the regressors and regressand
   yn <- column.name
   if(fn == 'P') { xn <- GDP.growth.percent.name; d <- 'GDP' }
   if(fn == 'I') { xn <- GDI.growth.percent.name; d <- 'GDI' }
   
   # - for the oytput stream ...
   print('====='); print(d); print('=====')
   
   # - get rid of NA rows
   dlm <- df[!is.na(df[, GDI.growth.percent.name]), ] # limit the data set
   dlm <- dlm[!is.na(dlm[, xn]) & !is.na(dlm[, yn]), ] # vectorised and
   
   # - get the regression data and regress ...
   y <- dlm[, yn]
   x <- dlm[, xn]
   fit <- lm(y ~ x)
   print(summary(fit))
 
   # - Okun: y = mx + b; if y = 0 then x = -b/m
   b <- (fit$coefficients)[[1]]
   m <- (fit$coefficients)[[2]]
   okun <- round(-b/m, 3)
   b <- round(b, 3); m <- round(m, 3)
   
   # - Bayesian Information Criterion
   r <- residuals(fit)
   sigma.hat.residuals <- var(residuals(fit))
   T <- length(residuals(fit))
   k <- 2 # two coefficients
   BIC <- round(T * log(sigma.hat.residuals) + k * log(T), 3)
   
   # - R-squared
   r2 <- round(summary(fit)$r.squared, 3)

   # - plot  
   plotTitle <- paste("Okuns Rule (", d, " Growth v delta(UR) lagged ", lag,
    " Qs; R-sq=", r2, "; BIC=", BIC, ')', sep='')
   f <- paste(imageDir, p, '-', plotTitle, fileEnding, sep="")
   plot <- ggplot(data = dlm, mapping = aes_string(x=xn, y=yn)) + 
       geom_point() +
       geom_smooth(method=lm) +
       geom_hline(yintercept=0, colour='black', size=0.4, alpha=0.4) +
       ylab(paste(p, " change in Unemployment Rate (% points)", sep='')) +
       xlab(paste(p, ' change in ', d, ' (%)', sep='')) +
       labs(title = plotTitle)
   plot <- addFootnote(plot, paste('Source: ABS 5206 and 6202; Okun threshold: ',
    okun, '; T=', T, '; y-intercept=', b, '; slope=', m, sep=''))
   ggsave(plot, width=plotWidth, height=plotHeight, filename=f, dpi=dpiset)
   
   if(lag == 0)
   {
    # - an extra plot at lag=0
    dlm$okun <- okun
    breaks <- c('okun', xn)
    labels <- c('Okun', paste(p, d, 'Percent Growth', sep=' ') )
    gf <- data.frame(breaks=breaks, labels=labels)
    f <- paste(imageDir , p, '-', d, '-v-okun-',  sep='')
    chartMulti (df=dlm, graphFrame=gf, saveLocPrefix=f,
           plotText=list(heading=paste(p, ' Growth in ', d, sep=''),
            yLabel='Percent'), 
           footnote=paste('Source: ABS 5206 and 6202;  Okun: ',
            okun, sep='') )
   }
  }
  print('-------------------------------------------')
 }
}

### ----- tidy up when done
dbDisconnect(conn)

Productivity growth - a mixed story

I last looked at productivity growth in March. Since then we have had two quarters of National Accounts data, so I thought it time to update some of the charts. In terms of GDP per hour worked, productivity has grown over Q1 and Q2 in 2012.

In terms of growth in nominal Non-Farm GDP per dollar paid in Non-Farm Employee Compensation our productivity has declined over Q1 and Q2 2012.

Last time I only looked a the data in annual terms (summed four quarters non-farm GDP divided by summed four quarters non-farm employee compensation). So we will start with the annual charts.

I have also compared the quarterly data (GDP/Compensation) with the quarterly data for the same quarter last year. This quarterly data is a little more noisy, but it tells a more up to date story.

As with the summed annual data, the negative growth rate on this productivity measure is now lower than it has been since the early 1980s.

GDP, GDI and Okun

Yesterday, I noted that the recent jump in real gross domestic product (GDP) did not appear to be driving a fall in the unemployment rate, as would be expected under Okun's rule.  I wondered  whether real gross domestic income (GDI) - GDP adjusted for changes in the terms of trade - would provide a better explanation for the unemployment rate. To help answer that question I have completed a series of simple linear regressions.

First, however, a quick recap: Okun's rule can be expressed as a relationship between GDP growth and the change in the unemployment rate. While the Okun co-efficient varies from country to country, at its simplest the Australian data suggests that GDP typically needs to be growing faster than three per cent each year for the unemployment rate to decline year-on-year.

Because the GDI data is only available back to 1986, I have limited my all of my regression models to the available data since that date (the charts span the same period). I have also looked at the data in terms of quarterly changes (comparing the current quarter with the previous quarter) and through-the year changes (comparing this quarter with the same quarter last year).

The first case I looked at was GDP v unemployment rate on a quarterly basis. The R output from my regression model is in the grey box below. You can skip the grey box if this is not your thing. In summary, it says that while this regression model is statistically significant at the 5 per cent level, it only explains 10 per cent of the variance in the data.

Analysis of Variance Table

Response: dlm$UR.qtly.growth.dif
                     Df Sum Sq Mean Sq F value    Pr(>F)    
dlm$GDP.qtly.growth   1 1.1321 1.13207  12.724 0.0005517 ***
Residuals           102 9.0753 0.08897                      
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Call:
lm(formula = dlm$UR.qtly.growth.dif ~ dlm$GDP.qtly.growth, na.action = "na.exclude")

Residuals:
     Min       1Q   Median       3Q      Max 
-0.53668 -0.19291 -0.01702  0.13009  1.16620 

Coefficients:
                    Estimate Std. Error t value Pr(>|t|)    
(Intercept)          0.10929    0.04725   2.313 0.022720 *  
dlm$GDP.qtly.growth -0.16160    0.04530  -3.567 0.000552 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.2983 on 102 degrees of freedom
Multiple R-squared: 0.1109, Adjusted R-squared: 0.1022 
F-statistic: 12.72 on 1 and 102 DF,  p-value: 0.0005517 

My second case was GDI v unemployment rate on a quarterly basis. Again, we have a regression model that is statistically significant at the 5% level. This model, however, explains 20 per cent of the variance in the data.

Analysis of Variance Table

Response: dlm$UR.qtly.growth.dif
                     Df Sum Sq Mean Sq F value    Pr(>F)    
dlm$GDI.qtly.growth   1 2.0870 2.08704  26.215 1.445e-06 ***
Residuals           102 8.1203 0.07961                      
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Call:
lm(formula = dlm$UR.qtly.growth.dif ~ dlm$GDI.qtly.growth, na.action = "na.exclude")

Residuals:
     Min       1Q   Median       3Q      Max 
-0.54498 -0.20640 -0.04825  0.15066  0.92417 

Coefficients:
                    Estimate Std. Error t value Pr(>|t|)    
(Intercept)          0.13064    0.04083    3.20  0.00183 ** 
dlm$GDI.qtly.growth -0.16019    0.03129   -5.12 1.45e-06 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.2822 on 102 degrees of freedom
Multiple R-squared: 0.2045, Adjusted R-squared: 0.1967 
F-statistic: 26.22 on 1 and 102 DF,  p-value: 1.445e-06 

My third model was GDP v unemplotyment rate on an annual basis. Again, statistically significant at the 5 per cent level. This model explains some 48 per cent of the variance.

Analysis of Variance Table

Response: dlm$UR.tty.growth.dif
                    Df Sum Sq Mean Sq F value    Pr(>F)    
dlm$GDP.tty.growth   1 42.787  42.787  96.787 < 2.2e-16 ***
Residuals          102 45.092   0.442                      
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Call:
lm(formula = dlm$UR.tty.growth.dif ~ dlm$GDP.tty.growth, na.action = "na.exclude")

Residuals:
     Min       1Q   Median       3Q      Max 
-1.70934 -0.44203 -0.02494  0.51764  1.43543 

Coefficients:
                   Estimate Std. Error t value Pr(>|t|)    
(Intercept)         1.25296    0.15237   8.223 6.69e-13 ***
dlm$GDP.tty.growth -0.41343    0.04202  -9.838  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.6649 on 102 degrees of freedom
Multiple R-squared: 0.4869, Adjusted R-squared: 0.4819 
F-statistic: 96.79 on 1 and 102 DF,  p-value: < 2.2e-16 

The final model is GDI v the unemployment rate on a through the year basis. Again, we have a valid regression, this time one that explains 56 per cent of the variance in the data. Of note: the gradient of the TTY-GDI model (-0.303) is not as steep as the gradient of the TTY-GDP model (-0.413). However, the average requirement for an unemployment rate that is falling over the year is a GDI through the year rate of 3.5 per cent.

Analysis of Variance Table

Response: dlm$UR.tty.growth.dif
                    Df Sum Sq Mean Sq F value    Pr(>F)    
dlm$GDI.tty.growth   1 49.720  49.720   132.9 < 2.2e-16 ***
Residuals          102 38.159   0.374                      
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Call:
lm(formula = dlm$UR.tty.growth.dif ~ dlm$GDI.tty.growth, na.action = "na.exclude")

Residuals:
     Min       1Q   Median       3Q      Max 
-1.64366 -0.40215  0.00907  0.42876  1.34243 

Coefficients:
                   Estimate Std. Error t value Pr(>|t|)    
(Intercept)         1.07347    0.11829   9.075 9.05e-15 ***
dlm$GDI.tty.growth -0.30252    0.02624 -11.528  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.6116 on 102 degrees of freedom
Multiple R-squared: 0.5658, Adjusted R-squared: 0.5615 
F-statistic: 132.9 on 1 and 102 DF,  p-value: < 2.2e-16 

Conclusion: for Australia at least, on both a quarterly basis and an annual basis, the GDI growth rate appears better at explaining changes in the unemployment rate than the GDP growth rate.

Caveat: I should admit to being a little troubled by the through the year regression models. I suspect the R-squared is over stated, as the sequential observations are not independent of each other (they include a 9 month period in common). Data points six months apart have a six month period in common. And you can do the math for data points that are nine months apart.

The next chart has the underlying data for the two through-the-year scatter plots.

GDP is currently growing at a rate that should see the unemployment rate decline. GDI is more lacklustre, growing at a much slower rate, which on average would see the unemployment rate grow.

GDI and jobs: are we up for a 7% contraction in the economy

The main statistic we use to measure the size of the economy is Gross Domestic Product. The ABS defines GDP as:

The total market value of goods and services produced in Australia after deducting the cost of goods and services used up (intermediate consumption) in the process of production, but before deducting allowances for the consumption of fixed capital (depreciation). It is equivalent to gross national expenditure plus exports of goods and services less imports of goods and services.

But GDP is not the only indicator of the size of the national economy. Real Gross Domestic Income (GDI) is another. According to the ABS, the real purchasing power of income generated by domestic production is affected by changes in import and export prices. Real gross domestic income adjusts the chain volume measure of GDP for the terms of trade effect. Some argue that GDI is a more accurate measure of the size of our economy than GDP.

Let's look at some charts.

What to make of all this?

I am going to theorise that there is a long-run equilibrium relationship between GDP and GDI. Statisticians have a lovely technical term for this kind of relationship: cointegration.  In the long-run real GDP lies some $15 to $20 billion above GDI. Up until the early noughties, both series grew at a similar pace. Since mid 2002, the start of the terms of trade boom, this relationship has broken down. Another chart.

 
If I am right, and these series are cointegrated in the long-run, then over time as the ToT boom unwinds we can expect the GDP-GDI relationship to return to its equilibrium difference relationship (that existed prior to mid-2002). This will mean that something like $100 billion to $120 billion would be removed from our annual $1.4 trillion national income (around 7%).

That has got to hurt. I suspect it is already hitting employment. Since  2011 the GDI growth rate has declined significantly and the number of jobs in the economy has stagnated (not withstanding a GDP growth bounce at the same time). From Okun, a GDP bounce would normally be correlated with an unemployment fall. 

If real GDI starts to actually decline, I suspect we will see the number of people in jobs decline. We will see government revenues decline (or at least slow in growth). Ultimately it may even challenge wage growth that exceeds inflation.

It is not going to be pretty as we learn to live on a reduced nation income.